Torch-nn学习: Simpley Layer

1.Linear:y = Ax + b

module = nn.Linear(inputDimension, outputDimension, [bias = true])

 module = nn.Linear(10, 5)  -- 10 inputs, 5 outputs

 print(module.weight)	//W
 print(module.bias)	//b

 print(module.gradWeight)
 print(module.gradBias)

2.SparseLinear: y = Ax + b

module = nn.SparseLinear(10000, 2)  -- 10000 inputs, 2 outputs

x = torch.Tensor({ {
     1, 0.1}, {
     2, 0.3}, {
     10, 0.3}, {
     31, 0.2} })	//dim not larger than 10000

3.Bilinear: forall k: y_k = x_1 A_k x_2 + b

module = nn.Bilinear(inputDimension1, inputDimension2, outputDimension, [bias = true])

4.Dropout:前向跟后向作用于相同位置，缩放了1/(1 - P)

module = nn.Dropout(p)

> module:forward(x)
  0   4   0   0
 10  12   0  16
[torch.DoubleTensor of dimension 2x4]

> module:backward(x, x:clone():fill(1))
 0  2  0  0
 2  2  0  2
[torch.DoubleTensor of dimension 2x4]

5.SpatialDropout:

module = nn.SpatialDropout(p)