Deep Learning With Pytorch

Deep Learning for Computer Vision Chapter05

Tensor的存储形式

在这里插入图片描述
4×3×4表示有三维,从外往里一层一层的展开,得到如上所示的矩阵。

view方法

view(x1,x2,x3,…xn),最后的n为几就会把原来的Tensor展成多少维的Tensor,当xj=-1时(j=1,2,3,…n),会自动适应每一个维度有多少个元素

import torch
x = torch.randn(4,3,4)
print(f'x:4×3×4\n{x}')
y = x.view(-1,4,2,1,1)
print(f'y:{y}')

使用view方法重置矩阵

def view_():
    x = torch.randn(2,2,2)
    print(f'x:\n{x}')
    y = x.view(2,1, 4)
    print(f'y:\n{y}')
    pass

输出的结果之一

x:
tensor([[[-0.3671, -1.1838],
         [-0.8485,  0.6301]],

        [[-1.1141,  1.1753],
         [ 0.6155,  1.2421]]])
y:
tensor([[[-0.3671, -1.1838, -0.8485,  0.6301]],

        [[-1.1141,  1.1753,  0.6155,  1.2421]]])

从结果可以看到,view方法将2×2×2的矩阵转换成了2×1×4的矩阵,它可以用下图来解释(代码里的数字是随机生成的,里面的数字可以更改):
在这里插入图片描述

Linux相关知识

对 -rw-r–r-- 或drwxrwxr-x等的解释

r:读权限
w:写权限
x:执行权限
u:文件拥有者
g:跟拥有者同组的用户
o:其他用户
a:所有用户,等同于ugo

在这里插入图片描述

Conv1d
class torch.nn.Conv1d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True)
相关参数解释:
in_channels(int) – 输入信号的通道。在文本分类中,即为词向量的维度
out_channels(int) – 卷积产生的通道。有多少个out_channels,就需要多少个1维卷积
kernel_size(int or tuple) - 卷积核的尺寸,卷积核的大小为(k,),第二个维度是由in_channels来决定的,所以实际上卷积大小为kernel_size*in_channels
stride(int or tuple, optional) - 卷积步长
padding (int or tuple, optional)- 输入的每一条边补充0的层数
dilation(int or tuple, `optional``) – 卷积核元素之间的间距
groups(int, optional) – 从输入通道到输出通道的阻塞连接数
bias(bool, optional) - 如果bias=True,添加偏置
#  卷积太难懂了,还是没看懂(前面也有不懂的,忘标注了)
#  输出通道到底是个什么意思?怎么输出的?

示例

def one_stride():                                                                                                                                                               
    conv = nn.Conv1d(1, 1, 3, bias=False)                                                                                                                                       
    sample = torch.randn(2, 1, 7)                                                                                                                                               
    print(f'sample:\n{sample}')                                                                                                                                                 
    print(conv.weight)                                                                                                                                                          
    # v_sample = conv(Variable(sample))       # 这里不能加Variable,否则出错:RuntimeError: bool value of Tensor with more than one value is ambiguous                                     
    v_sample = conv(sample)       # 这里不能加Variable,否则出错:RuntimeError: bool value of Tensor with more than one value is ambiguous                                                 
    print(f'v_sample:\n{v_sample}')                                                                                                                                             
    # v2_sample = conv(variable(sample))    # 但是可以加variable,但会有警告,显示:warnings.warn("torch.autograd.variable(...) is deprecated, use torch.tensor(...) instead")                 
    # print(f'v2_samole\n{v2_sample}')                                                                                                                                          
    # print(f'uuid.uuid4:{uuid.uuid4()}')     # uuid 通用唯一标识符                                                                                                                    
    pass                                                                                                                                                                                                                                                                                                                                                                                                                                                                             

输出结果

sample:
tensor([[[-0.7360,  0.7376,  0.2574, -1.3720,  0.6521,  1.0725, -0.6574]],

        [[ 0.9119,  1.1321,  0.1651, -0.6168,  0.6318, -0.1430,  1.8921]]])
Parameter containing:
tensor([[[ 0.4240, -0.3806, -0.4238]]], requires_grad=True)
v_sample:
tensor([[[-0.7019,  0.7962,  0.3550, -1.2845,  0.1469]],

        [[-0.1142,  0.6785,  0.0370, -0.4413, -0.4797]]],
       grad_fn=<SqueezeBackward1>)

Process finished with exit code 0
Padding

padding的用途: 保持边界信息;可以对有差异的图片进行补齐,使得图像的输入大小一致;在卷积层中加入padding ,会使卷基层的输入维度与输出维度一致; 同时,可以保持边界信息.

类型问题

transforms.Normalize((0.1307,), (0.3081,))处掉了一个逗号,导致程序无法运行。

transformation = transforms.Compose([transforms.ToTensor(),
                                         transforms.Normalize((0.1307,), (0.3081,))])   # https://stackoverflow.com/questions/56745486/pytorch-dataloader-indexerror-too-many-indices-for-tensor-of-dimension-0/56748549

以下是加逗号与不加逗号的区别:

def type_():
    print(f'type((0.5))\n{type((0.5))}')
    print(f'type((0.5,)):\n{type((0.5,))}')
    pass

输出结果

type((0.5))
<class 'float'>
type((0.5,)):
<class 'tuple'>
enumerate函数

enumerate() 函数用于将一个可遍历的数据对象(如列表、元组或字符串)组合为一个索引序列,同时列出数据和数据下标,一般用在 for 循环当中。
代码示例

def enumerate_():
    seasons = ['Spring','Summer','Fall','Winter']
    months = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']
    list_seasons = list(enumerate(seasons))
    list_months = list(enumerate(months))
    print(f'list_seasons:\n{list_seasons}')
    print(f'list_months:\n{list_months}')
    for i,month in enumerate(months):
        print(f'i:{i+1} month:{month}')
    pass

输出结果

list_seasons:
[(0, 'Spring'), (1, 'Summer'), (2, 'Fall'), (3, 'Winter')]
list_months:
[(0, 'Jan'), (1, 'Feb'), (2, 'Mar'), (3, 'Apr'), (4, 'May'), (5, 'Jun'), (6, 'Jul'), (7, 'Aug'), (8, 'Sep'), (9, 'Oct'), (10, 'Nov'), (11, 'Dec')]
i:1 month:Jan
i:2 month:Feb
i:3 month:Mar
i:4 month:Apr
i:5 month:May
i:6 month:Jun
i:7 month:Jul
i:8 month:Aug
i:9 month:Sep
i:10 month:Oct
i:11 month:Nov
i:12 month:Dec

AttributeError: module ‘torch.optim.optimizer’ has no attribute ‘zero_grad’

出现上面错误的原因是,没有明确具体的优化器,书中的代码把声明optimizer = optim.SGD(model.parameters(),lr = 0.01,momentum=0.5)放到了main()函数里,但具体使用是在自定义的fit()函数中,故出现上述错误。

一个问题又一个问题,啊啊啊啊啊,主要是因为代码没有说明需要导入的包,然后导致引入的包出错,问题一直找不出来;总结from torch.autograd import Variable这个包很重要;# # from distributed import Variable这个包很容易出错;代码还没弄懂,然后复现代码就出了这么多的错,真的是闹要疯了嘛!!!好在结果现在在出来,代码算是能复现了😭😭😭
在这里插入图片描述
在等结果ing
20次迭代后的运行结果如下

training loss is  0.64 and training accuracy is 47688/60000     79.48
validation loss is  0.13 and validation accuracy is 9632/10000     96.32
training loss is   0.3 and training accuracy is 54637/60000     91.06
validation loss is 0.082 and validation accuracy is 9732/10000     97.32
training loss is  0.24 and training accuracy is 55737/60000     92.89
validation loss is 0.071 and validation accuracy is 9770/10000      97.7
training loss is  0.21 and training accuracy is 56311/60000     93.85
validation loss is 0.063 and validation accuracy is 9803/10000     98.03
training loss is  0.19 and training accuracy is 56590/60000     94.32
validation loss is 0.052 and validation accuracy is 9833/10000     98.33
training loss is  0.18 and training accuracy is 56875/60000     94.79
validation loss is 0.051 and validation accuracy is 9842/10000     98.42
training loss is  0.17 and training accuracy is 57058/60000      95.1
validation loss is 0.047 and validation accuracy is 9844/10000     98.44
training loss is  0.16 and training accuracy is 57182/60000      95.3
validation loss is 0.046 and validation accuracy is 9855/10000     98.55
training loss is  0.16 and training accuracy is 57242/60000      95.4
validation loss is 0.041 and validation accuracy is 9856/10000     98.56
training loss is  0.15 and training accuracy is 57371/60000     95.62
validation loss is  0.04 and validation accuracy is 9861/10000     98.61
training loss is  0.15 and training accuracy is 57441/60000     95.74
validation loss is 0.036 and validation accuracy is 9877/10000     98.77
training loss is  0.14 and training accuracy is 57526/60000     95.88
validation loss is 0.034 and validation accuracy is 9888/10000     98.88
training loss is  0.14 and training accuracy is 57645/60000     96.07
validation loss is 0.036 and validation accuracy is 9877/10000     98.77
training loss is  0.13 and training accuracy is 57625/60000     96.04
validation loss is 0.035 and validation accuracy is 9876/10000     98.76
training loss is  0.13 and training accuracy is 57660/60000      96.1
validation loss is 0.035 and validation accuracy is 9885/10000     98.85
training loss is  0.13 and training accuracy is 57730/60000     96.22
validation loss is 0.034 and validation accuracy is 9888/10000     98.88
training loss is  0.13 and training accuracy is 57764/60000     96.27
validation loss is 0.037 and validation accuracy is 9884/10000     98.84
training loss is  0.12 and training accuracy is 57769/60000     96.28
validation loss is 0.035 and validation accuracy is 9885/10000     98.85
training loss is  0.12 and training accuracy is 57788/60000     96.31
validation loss is 0.037 and validation accuracy is 9878/10000     98.78

Process finished with exit code 0

完整代码实现20次的损失及精确度的迭代优化计算,有一些是测试的(被注释的部分)

import torch
from glob import glob
import os
import numpy as np
import matplotlib.pyplot as plt
import shutil
from torchvision import transforms, datasets
from torchvision import models
from torch.autograd import Variable
import torch.nn as nn
import torch.nn.functional as F
from torch.optim import lr_scheduler, optimizer
from torch import optim
from torchvision.datasets import ImageFolder
from torchvision.utils import make_grid
from torch.utils.data import Dataset,DataLoader
import time


class Net(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv1 = nn.Conv2d(1, 10, kernel_size=5)
        self.conv2 = nn.Conv2d(10, 20, kernel_size=5)
        self.conv2_drop = nn.Dropout2d()
        self.fc1 = nn.Linear(320, 50)
        self.fc2 = nn.Linear(50, 10)
        pass

    def forward(self, x):
        x = F.relu(F.max_pool2d(self.conv1(x), 2))
        x = F.relu(F.max_pool2d(self.conv2_drop(self.conv2(x)), 2))
        x = x.view(-1, 320)
        x = F.relu(self.fc1(x))
        x = F.dropout(x, training=self.training)
        x = self.fc2(x)
        return F.log_softmax(x)



def fit(epoch, model, data_loader, phase='training', volatile=False):
    optimizer = optim.SGD(model.parameters(),lr = 0.01,momentum=0.5)

    if phase == 'training':
        model.train()
    if phase == 'validation':
        model.eval()
        volatile = True
    running_loss = 0.0
    running_correct = 0
    for batch_idx, (data, target) in enumerate(data_loader):

        data, target = Variable(data, volatile), Variable(target)
        if phase == 'training':
            optimizer.zero_grad()
        output = model(data)
        loss = F.nll_loss(output, target)

        running_loss += F.nll_loss(output, target, size_average=False).data
        preds = output.data.max(dim=1, keepdim=True)[1]
        running_correct += preds.eq(target.data.view_as(preds)).cpu().sum()
        if phase == 'training':
            loss.backward()
            optimizer.step()

    loss = running_loss / len(data_loader.dataset)
    accuracy = 100. * running_correct / len(data_loader.dataset)

    print(
        f'{phase} loss is {loss:{5}.{2}} and {phase} accuracy is {running_correct}/{len(data_loader.dataset)}{accuracy:{10}.{4}}')
    return loss, accuracy
def print_and_plot(model,train_loader,test_loader):
    train_losses, train_accuracy = [], []
    val_losses, val_accuracy = [], []
    for epoch in range(1, 20):
        epoch_loss, epoch_accuracy = \
            fit(epoch, model, train_loader, phase='training', volatile=False)
        val_epoch_loss, val_epoch_accuracy = \
            fit(epoch, model, test_loader, phase='validation', volatile=False)
        train_losses.append(epoch_loss)
        print(f'train_loss:\n{train_losses}')
        train_accuracy.append(epoch_accuracy)
        val_losses.append(val_epoch_loss)
        val_accuracy.append(val_epoch_accuracy)
    plt.figure(1)
    # plot the trainging and test loss
    plt.plot(range(1, len(train_losses) + 1), train_losses, 'bo', label='training loss')
    plt.plot(range(1, len(val_losses) + 1), val_losses, 'r', label='validation loss')
    plt.legend()
    plt.savefig('MNIST' + str(epoch) + '.jpg')
    plt.figure(2)
    # plots the training and test accuracy
    plt.plot(range(1, len(train_accuracy) + 1), train_accuracy, 'bo', label='training accuracy')
    plt.plot(range(1, len(val_accuracy) + 1), val_accuracy, 'r', label='val accuracy')
    plt.legend()
    plt.savefig('MNIST_accuracy')

    pass

def main():
    dir = '/home/ZhangXueLiang/LiMiao/dataset/Mnist'
    model = Net()
    transformation = transforms.Compose([transforms.ToTensor(),
                                         transforms.Normalize((0.1307,), (
                                             0.3081,))])  # https://stackoverflow.com/questions/56745486/pytorch-dataloader-indexerror-too-many-indices-for-tensor-of-dimension-0/56748549
    train_dataset = datasets.MNIST(dir, train=True, transform=transformation, download=True)
    test_dataset = datasets.MNIST(dir, train=False, transform=transformation, download=True)

    train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)
    test_loader = torch.utils.data.DataLoader(test_dataset, batch_size=32, shuffle=True)
    print_and_plot(model, train_loader, test_loader)
    # view_()
    # variable_()
    # type_()
    # enumerate_()
    pass
def enumerate_():
    seasons = ['Spring','Summer','Fall','Winter']
    months = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']
    list_seasons = list(enumerate(seasons))
    list_months = list(enumerate(months))
    print(f'list_seasons:\n{list_seasons}')
    print(f'list_months:\n{list_months}')
    for i,month in enumerate(months):
        print(f'i:{i+1} month:{month}')

    pass

def variable_():

    # # 对数求导
    # x = Variable(torch.Tensor([1]), requires_grad = True)
    # w = Variable(torch.Tensor([3]), requires_grad = True)
    # b = Variable(torch.Tensor([3]), requires_grad = True)
    # # Bulid a computational graph
    # y = w * x + b # y = 2*x +3
    # y.backward()  #y.backward()这一行代码就是所谓的自动求导,直接通过这个就可以对所有需要梯度的变量进行求导,得到他们的梯度
    # print(f'x.grad:\n{x.grad}')
    # print(f'w.grad:\n{w.grad}')
    # print(f'b.grad:\n{b.grad}')

    # 对矩阵求导
    x = torch.randn(3)
    x = Variable(x,requires_grad = True)
    w = Variable(torch.randn(3),requires_grad  =True)
    y = w.mul(x)  # torch.mul(a, b)是矩阵a和b对应位相乘,a和b的维度必须相等,比如a的维度是(1, 2),b的维度是(1, 2),返回的仍是(1, 2)的矩阵
                  # torch.mm(a, b)是矩阵a和b矩阵相乘,比如a的维度是(1, 2),b的维度是(2, 3),返回的就是(1, 3)的矩阵
    print(f'x:\n{x}')
    print(f'w:\n{w}')
    print(f'y:\n{y}')
    y.backward(torch.Tensor([1,0.1,0.01]))   # y.backward(torch.Tensor([1,0.1,0.01])) ,得到的梯度是他们原有的梯度乘以1 ,0.1 和0.01.
    print(f'x.grad:\n{x.grad}')
    print(f'w.grad:\n{w.grad}')


    pass

def type_():
    print(f'type((0.5))\n{type((0.5))}')
    print(f'type((0.5,)):\n{type((0.5,))}')
    pass

def view_():
    x = torch.randn(2 ,2 ,2)
    print(f'x:\n{x}')
    y = x.view(-1 ,1, 8)
    print(f'y:\n{y}')
    pass
if __name__ == '__main__':
    main()

训练结果用plot显示
但是和书上的训练结果差别很大,应该是shuffle随机划分之后导致数据集里的数据不同。
在这里插入图片描述
在这里插入图片描述
dogsandcats使用该模型的训练结果
在这里插入图片描述
在这里插入图片描述

torch.max(input, dim, keepdim=False, out=None)函数

返回一个命名元组(值,索引),其中值是给定维数dim中输入张量每一行的最大值,索引是找到的每个最大值(argmax)的索引位置。

如果keepdim为真,则输出张量的大小与输入相同,除非在dim维中它们的大小为1。否则,dim将被压缩 ,导致输出张量比输入张量少1维。
(dim的取值只能是 -2,-1,0,1)

def max_():
    x = torch.randn(4,4)
    print(f'x:\n{x}')
    y1 = torch.max(x,1)[0]
    y2 = torch.max(x,1)
    y3 = torch.max(x,1,keepdim=True)
    y4 = torch.max(x,-2)
    y5 = torch.max(x,-1)
    y6 = torch.max(x,0)
    print(f'y1:\n{y1}')
    print(f'y2:\n{y2}')
    print(f'y3:\n{y3}')
    print(f'y4:\n{y4}')
    print(f'y5:\n{y5}')
    print(f'y6:\n{y6}')

    # x:
    # tensor([[-0.7551, -2.6309, -2.4808, -1.0101],
    #         [ 0.6619, -0.2234, -1.0197, -1.2845],
    #         [ 1.3562,  0.6646, -0.6744, -0.4259],
    #         [-0.3586,  1.0980, -0.4761,  1.2448]])
    # max 输出后是一个二元组(values, indices) ,
    # y1:  # y1 = torch.max(x,1)[0],因为是0,故取的是values
    # tensor([-0.7551,  0.6619,  1.3562,  1.2448])
    # y2:  # y2 = torch.max(x,1),没有指明,故取的是全部值
    # torch.return_types.max(
    # values=tensor([-0.7551,  0.6619,  1.3562,  1.2448]),
    # indices=tensor([0, 0, 0, 3]))
    # y3:  # y3 = torch.max(x,1,keepdim=True),保持维度不变,故变成了4×1
    # torch.return_types.max(
    # values=tensor([[-0.7551],
    #         [ 0.6619],
    #         [ 1.3562],
    #         [ 1.2448]]),
    # indices=tensor([[0],
    #         [0],
    #         [0],
    #         [3]]))
    # y4:  # y4 = torch.max(x,-2),改变的是dim值,求的是列最大值
    # torch.return_types.max(
    # values=tensor([ 1.3562,  1.0980, -0.4761,  1.2448]),
    # indices=tensor([2, 3, 3, 3]))
    # y5:  # 同  y2 = torch.max(x,1),结果一样
    # torch.return_types.max(
    # values=tensor([-0.7551,  0.6619,  1.3562,  1.2448]),
    # indices=tensor([0, 0, 0, 3]))
    # y6:  # 同  y4 = torch.max(x,-2),结果一样
    # torch.return_types.max(
    # values=tensor([ 1.3562,  1.0980, -0.4761,  1.2448]),
    # indices=tensor([2, 3, 3, 3]))
    pass
损失函数的选择

之前选用的时loss = F.nll_loss(output, target)但输出结果为training loss is nan and training accuracy is 11148/23000 48.47,这是因为损失函数选择不当导致loss为nan,后改为cross_entropy,结果为training loss is 0.04 and training accuracy is 22666/23000 98.55.

VGG模型下的loss和accuracy

plot显示
在这里插入图片描述
在这里插入图片描述

Conv2d

CLASS torch.nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode=‘zeros’)
对二维卷积的一点点理解

# Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, dilation=1, groups=1, bias=True, padding_mode='zeros')        
# 二维卷积都是四维的,输入形式:(N,Cin,Hin,Win);输出形式: (N,Cout,Hout,Wout)                                                                           
# 其中Cin = in_channels,Cout = out_channels,N 根据需要确定,Hin,Win由公式确定 
# N is a batch size                         
# C denotes a number of channels            
# H is a height of input planes in pixels   
# W is width in pixels                       
# 如果需要加多层卷积的话,相邻卷积层之间的联系如下:上层卷积的 out_channels 必须和下层卷积的 in_channels 相同                                                                  
def conv2d_():                                                                                                                      
    m1 = nn.Conv2d(16,33,3,stride = 2)                                                                                              
    m2 = nn.Conv2d(16,33,(3,5),stride=(2,1),padding=(4,2))                                                                          
    m3 = nn.Conv2d(16,33,(3,5),stride=(2,1),padding=(4,2),dilation=(3,1))                                                           
    input = torch.randn(20,16,50,100)                                                                                               
    output1 = m1(input)                                                                                                             
    output2 = m2(input)                                                                                                             
    output3 = m3(input)                                                                                                             
    print(f'output1:\n{output1}')                                                                                                   
    print(f'output2:\n{output2}')                                                                                                   
    print(f'output3:\n{output3}')                                                                                                   
    pass                                                                                                                                                                                                                                                             
  • 0
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值