卷积神经网络调参经验

实习遇到的问题吧 二值化网络加入XNor操作后，精度下降严重，更改第一层最后一层不使用Xnor，问题仍未解决，一时无从下手。

犹犹豫豫看到一篇paper
How to Train a Compact Binary Neural Network with High Accuracy

To consolidate our analysis, we have the statistics on how
many weights are changing their signs under different learning
rates during BNN and full precision network training.
The results are shown in Figure 2. It can be observed that,
under the same learning rate of 0.01, the sign changes for
BNN is nearly 3 orders of magnitude larger than that of a
full precision network. Only when the learning rate of BNN
is lowered to 0.0001, the two results become close.
Hence we conclude that a lower learning rate is more
preferred for training BNN to avoid frequent sign changes.
Our experiments show that when the learning rate is lowered
to 0.0001, a tremendous accuracy gain is obtained and
BNN