【Pytorch函数总结大全】不定期更新~

发现自己干的事情和兔叭咯似的，沉迷挖坑

函数总结

• torch.nn.Identity(): 返回原输入，用于保持网络层数
• torch.nn.MoudleList(): 构建一个存放layer，module的list，有利于forward的简写
  -append和extend可以添加model进入list
  -e.g.

class model2(nn.Module):
    def __init__(self):
        super(model2, self).__init__()
        self.layers=nn.ModuleList([
            nn.Linear(1,10), nn.ReLU(),
            nn.Linear(10,100),nn.ReLU(),
            nn.Linear(100,10),nn.ReLU(),
            nn.Linear(10,1)])
    def forward(self,x):
        out=x
        for i,layer in enumerate(self.layers):
            out=layer(out)
        return out
        
self.linears.extend([nn.Linear(size1, size2) for i in range(1, num_layers)])

self.linears.append(nn.Linear(size1, size2)

• torch.nn.Sequential()：与modulelist类似，添加之后可以是做一个整体
  -e.g.

class model3(nn.Module):
    def __init__(self):
        super(model3, self).__init__()
        self.network=nn.Sequential(
            nn.Linear(1,10),nn.ReLU(),
            nn.Linear(10,100),nn.ReLU(),
            nn.Linear(100,10),nn.ReLU(),
            nn.Linear(10,1)
        )
    def forward(self, x):
        return self.network(x)
        
#可以在创建之后加入新的模型，而且可以给子模型定义唯一的名称索引，方便获取        
self.network.add_module("linear1",nn.Linear(100,100))
#获得这个子模型
linear=self.network.linear1

#另一种写法
from collections import OrderedDict
self.network=nn.Sequential(OrderedDict(
            ("linear1",nn.Linear(1,10)),
            ("activation1",nn.ReLU()),
            ("linear2",nn.Linear(10,100))
        ))

• model.to(device): 将模型加载到指定设备上
• optimizer.zero_grad()：梯度清零， 通过grad accumulate可以节约显存

for i,(image, label) in enumerate(train_loader):
    # 1. input output
    pred = model(image)
    loss = criterion(pred, label)

    # 2.1 loss regularization
    loss = loss / accumulation_steps  
 
    # 2.2 back propagation
    loss.backward()

    # 3. update parameters of net
    if (i+1) % accumulation_steps == 0:
        # optimizer the net
        optimizer.step()        # update parameters of net
        optimizer.zero_grad()   # reset gradient

• loss = criterion(preds, targets) ：求解loss
• loss.backward() ：反向传播求解梯度
• optimizer.step()：更新模型参数
• with torch.no_grad(): warp内的计算过程不会被torch track