PyTorch一些小知识点

优化

filter lambda

torch.optim.SGD(filter(lambda p: p.requires_grad, model_parameters), lr=lr, momentum=momentum,weight_decay=weight_decay)

1. lambda 是匿名函数 ：lambda arg : expression
arg是参数，expression 是参数的表达式
2. filter 用于过滤序列：filter(function, iterable)；function – 判断函数；iterable被判断的对象
3. 每个变量requires_grad标志，当requires_grad=Fasle时不需要更新梯度， 适用于冻结某些层的梯度
4. optim.SGD(model.parameters(), lr = 0.01, momentum=0.9)

所以上式的意思为：给定模型的参数model_parameters，利用filter排除所有requires_grad=Fasle，即不需要更新梯度的参数。对需要更新梯度的参数进行SGD
参考文献：lambda表达式和filter函数; Python filter() 函数 ； 自动求导机制requires_grad

模型的存储与下载

数据集相关

pytorch torchvision transform

train_img_shape =(1024, 512) # type=tuple
img_transform_list = [
    Scale((1024, 512), Image.BILINEAR), #  Image.BILINEAR双线性缩放
    ToTensor(),
    Normalize([.485, .456, .406], [.229, .224, .225])
    RandomRotation(),
    RandomHorizontalFlip(),
    RandomSizedCrop()
 ]
img_transform = Compose(img_transform_list)

其中

1. torchvision.transforms.Scale(size, interpolation=2)是按照规定的尺寸重新调节PIL.Image
2. ToTensor()：是将 PIL Image or numpy.ndarray格式转化为 tensor（pytoch能处理的张量格式）
3. class torchvision.transforms.Normalize(mean, std)给定均值：(R,G,B) 方差：（R，G，B），将会把Tensor正则化。mean (sequence)– 序列R, G, B的均值。
   std (sequence) – 序列 R, G, B 的平均标准偏差.
4. class torchvision.transforms.RandomRotation(degrees, resample=False, expand=False, center=None)功能：依degrees随机旋转一定角度 [参考文献](https://blog.csdn.net/weixin_38533896/article/details/860285095.
5. class torchvision.transforms.RandomHorizontalFlip(p=0.5)功能：依据概率p对PIL图片进行水平翻转
6. class torchvision.transforms.RandomSizedCrop(size, interpolation=2)
   先将给定的PIL.Image随机切，然后再resize成给定的size大小。
7. Compose：对输入图像img执行所有的transform操作
8. torch.long(): 将tensor投射为long类型 newtensor = tensor.long()

src_dataset = get_dataset(dataset_name=gta, split=train, img_transform=img_transform,
                          label_transform=label_transform, test=False, input_ch=3)
                          
def get_dataset(dataset_name, split, img_transform, label_transform, test, input_ch=3):
    assert dataset_name in ["gta", "city", "test", "city16", "synthia"]

    name2obj = {
        "gta": GTADataSet,
        "city": CityDataSet,
        "city16": CityDataSet,
        "synthia": SynthiaDataSet,
    }
    ##Note fill in the blank below !! "gta....fill the directory over images folder.
    name2root = {
        "gta": "/data/ysh/data/GTA5/",  ## Fill the directory over images folder. put train.txt, val.txt in this folder
        "city": "/data/ysh/data/Cityscapes/",  ## ex, ./www.cityscapes-dataset.com/file-handling
        "city16": "",  ## Same as city
        "synthia": "",  ## synthia/RAND_CITYSCAPES",
    }
    dataset_obj = name2obj[dataset_name]
    root = name2root[dataset_name]

    if dataset_name == "city16":
       略
    elif dataset_name == "gta":
        return dataset_obj(root=root, split=split, img_transform=img_transform, label_transform=label_transform,
                       test=test, input_ch = input_ch)
    # print dataset_name
    else:
        return dataset_obj(root=root, split=split, img_transform=img_transform, label_transform=label_transform,
                       test=test, input_ch = input_ch)

dataset_obj=GTADataSet-
root="/data/ysh/data/GTA5/"

torchsummary 打印 Pytorch 模型

安装

pip install torchsummary
#或
git clone https://github.com/sksq96/pytorch-summary

使用方法

from torchsummary import summary
summary(pytorch_model, input_size=(channels, H, W))

其中， input_size 是必须设定的，以对网络进行一次 forward.

使用例示
CNN for MNSIT
import torch
import torch.nn as nn
import torch.nn.functional as F
from torchsummary import summary

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x, dim=1)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # PyTorch v0.4.0
model = Net().to(device)

summary(model, (1, 28, 28))

输出结果：

---

        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1           [-1, 10, 24, 24]             260
            Conv2d-2             [-1, 20, 8, 8]           5,020
         Dropout2d-3             [-1, 20, 8, 8]               0
            Linear-4                   [-1, 50]          16,050
            Linear-5                   [-1, 10]             510
================================================================
Total params: 21,840
Trainable params: 21,840
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.06
Params size (MB): 0.08
Estimated Total Size (MB): 0.15
----------------------------------------------------------------