Caffe系列（12）利用caffe.proto自定义自己的网络

最新推荐文章于 2019-04-30 15:21:46 发布

2014wzy

最新推荐文章于 2019-04-30 15:21:46 发布

阅读量1.1k

点赞数

分类专栏： caffe框架

caffe框架专栏收录该内容

62 篇文章 1 订阅

订阅专栏

利用caffe.proto自定义自己的网络

利用caffeproto自定义自己的网络

caffe.proto存放在src/caffe/proto/目录下，建议读者能够打开它，跟着这篇教程，学会如何把它用起来（很多人都说要看，反正才一千三百多行是吧…小编我看了两百行就想睡觉了，关键是看了还容易忘，所以小编的建议是在需要用它的时候再去看它）。

先不急，让我们来看一下caffe.proto里面都是些什么

对于懒得在翻目录的读者，可以点击这个链接，不过还是建议读者都看本地的，毕竟以后会经常用到的
https://github.com/BVLC/caffe/blob/master/src/caffe/proto/caffe.proto

下面列出文件里面的前十几行= =

<code class="language-protobuf hljs vala has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">message BlobShape {
  repeated <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int64</span> dim = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> [packed = <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>];
}

message BlobProto {
  optional BlobShape shape = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>;
  repeated <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">float</span> data = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span> [packed = <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>];
  repeated <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">float</span> diff = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span> [packed = <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>];
  repeated <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">double</span> double_data = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span> [packed = <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>];
  repeated <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">double</span> double_diff = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span> [packed = <span class="hljs-literal" style="color: rgb(0, 102, 102); box-sizing: border-box;">true</span>];

  <span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// 4D dimensions -- deprecated.  Use "shape" instead.</span>
  optional <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int32</span> num = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span> [<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">default</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>];
  optional <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int32</span> channels = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span> [<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">default</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>];
  optional <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int32</span> height = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span> [<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">default</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>];
  optional <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">int32</span> width = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span> [<span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">default</span> = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">0</span>];
}

<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// The BlobProtoVector is simply a way to pass multiple blobproto instances</span>
<span class="hljs-comment" style="color: rgb(136, 0, 0); box-sizing: border-box;">// around.</span>
message BlobProtoVector {
  repeated BlobProto blobs = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>;
}</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li><li style="box-sizing: border-box; padding: 0px 5px;">18</li><li style="box-sizing: border-box; padding: 0px 5px;">19</li><li style="box-sizing: border-box; padding: 0px 5px;">20</li><li style="box-sizing: border-box; padding: 0px 5px;">21</li><li style="box-sizing: border-box; padding: 0px 5px;">22</li><li style="box-sizing: border-box; padding: 0px 5px;">23</li></ul>

相信读者看到上面的内容会感到很熟眼，这不就是Blob吗？对，这就是Blob的细节，对于一直搞不懂Blob里面放的是什么东西的读者，现在可以深入到内部去理解Blob了。

读者在看.proto是会经常遇到deprecated这样的注释，建议读者跳过跟这个注释有关的代码，因为这些代码都是过时的，很有可能会在某一天就被删掉。

注意到这些定义有两种前缀，一个是optional，一个是repeated。repeated是数组，而optional就是一个数据。因此BlobShape里面包含了一个shape的数组。然后BlobProto里面就包含了5个数据（不考虑那些被标了deprecated的），分别是BlobShape，float和double类型的data和diff。因此，一个Blob里面的数据就是他自身的尺寸，以及用于前向的data和用于后向的diff，这样看够直接了吧？

然后我们再来看看Layer有关的，为了节约篇幅，下面只截取一部分

<code class="language-protobuf hljs r has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">message LayerParameter {
  optional string name = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>; // the layer name
  optional string type = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>; // the layer type
  repeated string bottom = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>; // the name of each bottom blob
  repeated string top = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">4</span>; // the name of each top blob
  optional Phase phase = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>;
  repeated float loss_weight = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">5</span>;
  repeated ParamSpec param = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">6</span>;
  repeated BlobProto blobs = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">7</span>;
  repeated bool propagate_down = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">11</span>;
  repeated NetStateRule include = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">8</span>;
  repeated NetStateRule exclude = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">9</span>;
  optional TransformationParameter transform_param = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">100</span>;
  optional LossParameter loss_param = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">101</span>;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span>
  optional ConvolutionParameter convolution_param = <span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">106</span>;
  <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">...</span></code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li><li style="box-sizing: border-box; padding: 0px 5px;">12</li><li style="box-sizing: border-box; padding: 0px 5px;">13</li><li style="box-sizing: border-box; padding: 0px 5px;">14</li><li style="box-sizing: border-box; padding: 0px 5px;">15</li><li style="box-sizing: border-box; padding: 0px 5px;">16</li><li style="box-sizing: border-box; padding: 0px 5px;">17</li></ul>

可以看到LayerParameter包含很东西，比如什么name啊，type啊，blabla。记性好的读者应该发现在我们的Caffe学习2那部分的Nets的layer的protobuf代码有的，LayerParameter也都有。因此不难猜测，如果想要使用Caffe的Layer，那么，caffe.proto的内容将会提供很大的帮助。

让我们把caffe.proto用起！

下面我们先给出一个caffe的net的定义的结构，方便我们往里面加东西。

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> layers <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> L, params <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> P

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">mynet</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(lmdb, batch_size)</span>:</span>
    n = caffe.NetSpec()

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> n.to_proto()</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li></ul>

上面的代码，我们一开始就import了两个包，一个是layers，另一个是params。layers里面包含了Caffe所以内置的层（比如卷积，ReLU等），而params则包含了各种枚举值。方法的参数中的lmdb是指Caffe支持的数据库的一种，叫lmdb，我们传入数据库的路径即可。而n=caffe.NetSpec()是获取Caffe的一个Net，我们只需不断的填充这个n，最后面把n输出到文件就会使我们在Caffe学习2里面看到的Net的protobuf的定义。
接下来，我们想要通过lmdb读取数据。那代码该怎么写呢？谷歌一下，或者是偷看caffe的示例程序，我们可以知道应该如下

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LMDB, source=lmdb, transform_param=dict(scale=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255</span>), ntop=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

由于lmdb里面存放了两个数据，一个是data，一个是label，因此这个层上面会有2个blobs，故ntop=2（小编试过把它删了，结果报错，看来这个还是少不了的）。
好，然后我们想添加一层Convolution layer，我们这次不谷歌也不看示例，看看怎么用caffe.proto写代码

首先，我们看下LayerParameter里面有没Convolution的身影，然后发现有ConvolutionParameter，这个就说明Caffe内置有Convolution Layer。然后在LayerParameter里面找到我们需要设置的参数，这里我们可以不用设置任何参数，python的接口会帮我们自动生成。因此我们有了我们的第一步代码

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">n.conv1 = L.Convolution(n.data, ...)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

然后我们搜ConvolutionParameter，找到改名下我们需要用到的一下参数，比如num_output（过滤器的数目），kernel_size(过滤器的展度)，pad（填充数），stride（步长），其中必不可少的num_output和kernel_size，这两个没有default值，因此如果少了的话Caffe会出错。
然后我们有了我们的完整代码

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">n.conv1 = L.Convolution(n.data, kernel_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

最后我们还想给这一层的权重一个比较科学的初始化，比如xavier。找到ConvolutionParameter的weight_filler，发现是一个FillerParameter，同理，找到FillerParameter下。我们只要把type设置正确即可。最终我们可以有如下的代码

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">n.conv1 = L.Convolution(n.data, kernel_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, weight_filler=({type:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"xavier"</span>}))</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li></ul>

同样地，我们还可以定义其他层（至于为什么就交给读者自己验证啦，有问题可以留言），比如

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;">n.score = L.InnerProduct(n.conv1, num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>, weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'xavier'</span>))
n.loss =  L.SoftmaxWithLoss(n.score, n.label)</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li></ul>

-最终我们用python定义了一个完整的Net

<code class="language-python hljs  has-numbering" style="display: block; padding: 0px; color: inherit; box-sizing: border-box; font-family: 'Source Code Pro', monospace;font-size:undefined; white-space: pre; border-radius: 0px; word-wrap: normal; background: transparent;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">from</span> caffe <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">import</span> layers <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> L, params <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">as</span> P

<span class="hljs-function" style="box-sizing: border-box;"><span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">def</span> <span class="hljs-title" style="box-sizing: border-box;">mynet</span><span class="hljs-params" style="color: rgb(102, 0, 102); box-sizing: border-box;">(lmdb, batch_size)</span>:</span>
    n = caffe.NetSpec()

    n.data, n.label = L.Data(batch_size=batch_size, backend=P.Data.LMDB, source=lmdb, transform_param=dict(scale=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1.</span>/<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">255</span>), ntop=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">2</span>)
    n.conv1 = L.Convolution(n.data, kernel_size=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">3</span>, num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">36</span>, pad=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">1</span>, weight_filler=({type:<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">"xavier"</span>}))
    n.score = L.InnerProduct(n.conv1, num_output=<span class="hljs-number" style="color: rgb(0, 102, 102); box-sizing: border-box;">10</span>, weight_filler=dict(type=<span class="hljs-string" style="color: rgb(0, 136, 0); box-sizing: border-box;">'xavier'</span>))
    n.loss =  L.SoftmaxWithLoss(n.score, n.label)

    <span class="hljs-keyword" style="color: rgb(0, 0, 136); box-sizing: border-box;">return</span> n.to_proto()</code><ul class="pre-numbering" style="box-sizing: border-box; position: absolute; width: 50px; top: 0px; left: 0px; margin: 0px; padding: 6px 0px 40px; border-right-width: 1px; border-right-style: solid; border-right-color: rgb(221, 221, 221); list-style: none; text-align: right; background-color: rgb(238, 238, 238);"><li style="box-sizing: border-box; padding: 0px 5px;">1</li><li style="box-sizing: border-box; padding: 0px 5px;">2</li><li style="box-sizing: border-box; padding: 0px 5px;">3</li><li style="box-sizing: border-box; padding: 0px 5px;">4</li><li style="box-sizing: border-box; padding: 0px 5px;">5</li><li style="box-sizing: border-box; padding: 0px 5px;">6</li><li style="box-sizing: border-box; padding: 0px 5px;">7</li><li style="box-sizing: border-box; padding: 0px 5px;">8</li><li style="box-sizing: border-box; padding: 0px 5px;">9</li><li style="box-sizing: border-box; padding: 0px 5px;">10</li><li style="box-sizing: border-box; padding: 0px 5px;">11</li></ul>