torch.nn.Dropout()细节记录

说明： 

1. 在model.train()模式下Dropout层是有效的，在model.eval()模式下，Dropout层是无效的。

2. torch.nn.Dropout是采用的inverted 版本，为了“保持期望不变”而进行了rescale(缩放），缩放比例是1/(1-p)，p是的丢弃比率。

Furthermore, the outputs are scaled by a factor of :math:`\frac{1}{1-p}` during
training. This means that during evaluation the module simply computes an
identity function.

#-*- coding:utf-8 -*-
#Author LJB Create on 2021/8/25
import torch.nn as nn
import torch
class Test(nn.Module):
    def __init__(self):
        super(Test,self).__init__()
        self.droprate=0.8
        self.dropout = nn.Dropout(self.droprate)

    def forward(self,x):
        o = self.dropout(x)
        print('++before dropout:',x/(1-self.droprate))
        print('---after dropout:',o)

T = Test()

data = torch.Tensor([1,2,3,4,5,6,7,8,9,10])
T.train()
for i in range(5):
    T(data)

print('#'*40)
T.eval()
for i in range(5):
    T(data)

运行结果：