对deeplearningToolBox的一点理解（SAE篇）

<pre name="code" class="cpp"><span style="font-family: Arial, Helvetica, sans-serif;">function test_example_SAE</span>

load mnist_uint8;

train_x = double(train_x)/255;
test_x  = double(test_x)/255;
train_y = double(train_y);
test_y  = double(test_y);        //将数据一开始初始化

%%  ex1 train a 100 hidden unit SDAE and use it to initialize a FFNN
%  Setup and train a stacked denoising autoencoder (SDAE)
rand('state',0)
sae = saesetup([784 100]);  

这里跳入saesetup函数，由函数可知返回的是sae的结构体

function sae = saesetup(size)
    for u = 2 : numel(size)   //numel(size)=2
        sae.ae{u-1} = nnsetup([size(u-1) size(u) size(u-1)]);  %size(1)=784 size(2)=100 size(3)=784
    end
end

这里调用了nnsetup函数，由该函数可知返回的也是nn结构体，可以看出训练后是把nn替代成sae.

function nn = nnsetup(architecture)
%NNSETUP creates a Feedforward Backpropagate Neural Network
% nn = nnsetup(architecture) returns an neural network structure with n=numel(architecture)
% layers, architecture being a n x 1 vector of layer sizes e.g. [784 100 10]

    nn.size   = architecture;   //architecture表示每一层由多少个神经元，总共有多少层(3)
    nn.n      = numel(nn.size);//网络层数3
    
    nn.activation_function              = 'tanh_opt';   %  Activation functions of hidden layers: 'sigm' (sigmoid) or 'tanh_opt' (optimal tanh).
    nn.learningRate                     = 2;            %  learning rate Note: typically needs to be lower when using 'sigm' activation function and non-normalized inputs.
    nn.momentum                         = 0.5;          %  Momentum
    nn.scaling_learningRate             = 1;            %  Scaling factor for the learning rate (each epoch)
    nn.weightPenaltyL2                  = 0;            %  L2 regularization
    nn.nonSparsityPenalty               = 0;            %  Non sparsity penalty
    nn.sparsityTarget                   = 0.05;         %  Sparsity target
    nn.inputZeroMaskedFraction          = 0;            %  Used for Denoising AutoEncoders
    nn.dropoutFraction                  = 0;            %  Dropout level (http://www.cs.toronto.edu/~hinton/absps/dropout.pdf)
    nn.testing                          = 0;            %  Internal variable. nntest sets this to one.
    nn.output                           = 'sigm';       %  output unit 'sigm' (=logistic), 'softmax' and 'linear'
    //对每一层的网络结构进行初始化，一共三个参数W,vW,p,其中W是主要的参数
    //vW是更新参数时的临时参数，p是所谓的sparsity,
    for i = 2 : nn.n   %生成两层权值和p{i}
        % weights and weight momentum
        nn.W{i - 1} = (rand(nn.size(i), nn.size(i - 1)+1) - 0.5) * 2 * 4 * sqrt(6 / (nn.size(i) + nn.size(i - 1)));   <span style="font-family: Arial, Helvetica, sans-serif;">//</span><span style="font-family: Arial, Helvetica, sans-serif;">随机取从-0.5到 2 * 4 * sq