Pytorch深度学习一些预备知识

1.“Numpy”的进阶版Tensor对象

Pytorch的Tensor（张量）和Numpy极度相似，但Tensor更强劲一点。其一是它能够被GPU运算；其二是可以自动求微分。

import torch
import numpy as np

Tensor可以定义在GPU上，也可以定义在CPU上，通常用dtype指定数据类型、device指定它的设别（是在CPU上还是在GPU上）

方法torch.Tensor与torch.tensor的不同

1.1 tensor与Tensor的不同

torch.Tensor(1).dtype

torch.float32

torch.tensor(1).dtype

torch.int64

大写是浮点型，小写是整型

1.2 张量的创建

# 用ndarray构建
tensor_ndarray = torch.tensor(np.array([[1,2],[3,4]]),dtype=torch.uint8)
tensor_ndarray

tensor([[1, 2],
        [3, 4]], dtype=torch.uint8)

# 用列表list创建
tensor_list = torch.tensor([[1,2],[3,4]],dtype = torch.float64)
tensor_list

tensor([[1., 2.],
        [3., 4.]], dtype=torch.float64)

#其他特殊tensor
uninitialized = torch.Tensor(3,2) # 初始化张量
rand_initialized = torch.rand(3,2) # 创建随机长量
matrix_with_ones = torch.ones(3,2) # 创建全为1的张张量
matrix_with_zeros = torch.zeros(3,2) # 创建全为0的张量

rand_initialized_size = rand_initialized.size()
rand_initialized_shape = rand_initialized.shape
# 判断是否相同
print(rand_initialized_size==rand_initialized_shape)
print(rand_initialized_shape[0])
print('='*30) # 分割线
print(rand_initialized_shape[1])

1.3 设备的转换

tensor_cuda = torch.device('cuda:0')
# c = torch.ones((2,2),device = tensor_cuda) 因为我的版本是CPU版本，没有GPU

通过device指定在GPU上定义变量之后，可以在终端上通过nvidia-smi命令查看显存占用情况。torch还支持在CPU版本与GPU版本之间的tensor转换与复制

# 如果你是GPU版本，这可以用下面的语句
c_cpu = c.to('cpu',torch.double)
c_gpu = tensor_list(tensor_cuda,torch.float)

1.4 数值运算

a_tensor = torch.tensor([[1,2],[3,4]],dtype = torch.int8)
print(a_tensor)
b_tensor = torch.tensor([[1,1],[1,1]],dtype = torch.int8)
print(b_tensor)

tensor([[1, 2],
        [3, 4]], dtype=torch.int8)
tensor([[1, 1],
        [1, 1]], dtype=torch.int8)

c_result = a_tensor * b_tensor
c_result # 注意这里的*表示元素与元素之间的乘法

tensor([[1, 2],
        [3, 4]], dtype=torch.int8)

# 如果要像矩阵乘法一样可以用‘@’、‘torch.mm(x,y)’、‘x.matmul(y)’
c_matmul = torch.mm(a_tensor,b_tensor)
c_matmul

tensor([[3, 3],
        [7, 7]], dtype=torch.int8)

# 数值运算
x = torch.ones(3,2)
print(x)
y = torch.ones(3,2)+2
print(y)
z = torch.ones(2,1)
print(z)
print('='*30)
print(x*y@z) #　ｘ*ｙ表示元素之间的点乘，＠ｚ，点乘的结果于ｚ进行矩阵的乘法

z = x.add(y)
print(z)
z = x.add_(y)
print(z)

print(x==z)
x = torch.rand(2,3)
y = torch.rand(3,4)
print(x.matmul(y))

1.5 一些特定功能的函数方法

• torch.clamp:将函数分段，可以用于去掉矩阵中过小、过大的元素
• torch.round:取整
• torch.tanh:计算双曲正切函数，即是将数值映射到[0,1]区间

a_practise = torch.tensor([[1.5,2],[3,4.1]],dtype = torch.float32)

torch.clamp(a_practise,min = 2,max = 3)

tensor([[2., 2.],
        [3., 3.]])

a_practise = torch.tensor([[1.5,2],[3,4.1]],dtype = torch.float32)
torch.round(a_practise)

tensor([[2., 2.],
        [3., 4.]])

torch.tanh(a_practise)

tensor([[0.9051, 0.9640],
        [0.9951, 0.9995]])

1.6 经常会用到，构建一些特殊的区间向量张量

torch.arange(4)

tensor([0, 1, 2, 3])

torch.arange(1,10,2)

tensor([1, 3, 5, 7, 9])

torch.linspace(1,2,10)

tensor([1.0000, 1.1111, 1.2222, 1.3333, 1.4444, 1.5556, 1.6667, 1.7778, 1.8889,
        2.0000])

## 1.7 补充产生随机张量的函数
torch.rand(5,5)

tensor([[0.6820, 0.6566, 0.6800, 0.9168, 0.8072],
        [0.8984, 0.3521, 0.3746, 0.4676, 0.7470],
        [0.1475, 0.0079, 0.8307, 0.3854, 0.9124],
        [0.1240, 0.0881, 0.2743, 0.7040, 0.6030],
        [0.9523, 0.3473, 0.9195, 0.7074, 0.2006]])

torch.randn(5,5)

tensor([[ 0.1802,  0.9702,  0.6747, -0.5608,  0.8149],
        [ 0.0238,  0.2082, -0.8594,  1.1675, -1.2505],
        [-1.7120,  0.2129,  0.1694,  0.8515, -0.3065],
        [ 0.2202,  1.1955,  1.4744,  0.0538,  1.1734],
        [ 0.0207,  0.4982,  0.5523, -0.2678,  0.8610]])

torch.randint(0,25,(5,5))

tensor([[10, 22, 19, 17, 21],
        [11,  1, 19,  7,  2],
        [18,  1, 18,  2, 12],
        [ 9, 11, 12,  2,  1],
        [13,  2, 24,  9,  3]])

1.8 最重要的索引和切片

a_tensor = torch.arange(9).view(3,3)
a_tensor

tensor([[0, 1, 2],
        [3, 4, 5],
        [6, 7, 8]])

# 基本索引
a_tensor[2,2]

tensor(8)

# 切片
a_tensor[1:,:-1] # 取第一行后的所有行，在进一步取除倒数第一列的所有列

tensor([[3, 4],
        [6, 7]])

# 带步长的切片 
a_tensor[::2] 

tensor([[0, 1, 2],
        [6, 7, 8]])

print(a_tensor[::1])

tensor([[0, 1, 2],
        [3, 4, 5],
        [6, 7, 8]])

print(a_tensor[:-1:1])

tensor([[0, 1, 2],
        [3, 4, 5]])

a_tensor[::-2]
# 注意负的步长，现在不被支持，会报错

---------------------------------------------------------------------------

ValueError                                Traceback (most recent call last)

<ipython-input-59-912197658ea5> in <module>
----> 1 a_tensor[::-2]
      2 # 注意负的步长，现在不被支持，会报错


ValueError: step must be greater than zero

# 整数索引
a_practise = torch.arange(9).view(3,3) 
row_no = [0,1]
col_no = [2,2]
print(a_practise[row_no,col_no])

# 布尔索引
print(a_practise[a_practise>4])

# torch.nonzero()用于返回非零值的索引矩阵
torch.nonzero(a_practise>4)


# torch.where(condition=XX , x, y) 判断condition的条件是否满足，满足则返回对应矩阵x相同位置的元素，否者返回y对应的元素
x = torch.randn(3,2)
y = torch.ones(3,2)
print(x)
print(y)
print(torch.where(x>0,x,y))

tensor([2, 5])
tensor([5, 6, 7, 8])
tensor([[-2.1295,  0.8144],
        [ 0.6349,  1.4752],
        [ 0.6596, -0.1794]])
tensor([[1., 1.],
        [1., 1.],
        [1., 1.]])
tensor([[1.0000, 0.8144],
        [0.6349, 1.4752],
        [0.6596, 1.0000]])

1.9 Tensor的变化、拼接、拆分

a = torch.rand(1,2,3,4,5)
print('元素个数：',a.nelement())
print('轴的个数：',a.ndimension())
print('矩阵维度：',a.size(),a.shape)

print(a.shape[4])
print('='*30) # 分割线

元素个数： 120
轴的个数： 5
矩阵维度： torch.Size([1, 2, 3, 4, 5]) torch.Size([1, 2, 3, 4, 5])
5
==============================

在Pytorch中，Tensor.reshape和Tensor.view都被用来改变Tensor的维度，较常会用前者， 后者因为要求对物理内存要求必须是连续的，同时它返回的是一个索引，如果更改返回值，则原始的数据一定被更改。但是reshape方法就不一定是引用还是复制操作。其他都是相同的。

b = a.view(2*3,4*5)
print(b)
print(b.shape)
print('='*30)
c = a.reshape(-1)
print(c)
print(c.shape)
print('='*30)
d = a.view(-1)
print(d)
print(d.shape)

tensor([[0.7733, 0.1794, 0.7046, 0.2685, 0.5017, 0.4303, 0.3463, 0.7707, 0.0760,
         0.2529, 0.8447, 0.8022, 0.6737, 0.0222, 0.1824, 0.4169, 0.2391, 0.5440,
         0.2884, 0.8852],
        [0.5118, 0.7946, 0.8106, 0.9128, 0.0094, 0.4233, 0.4691, 0.6234, 0.2335,
         0.9982, 0.3506, 0.7898, 0.9798, 0.9961, 0.5045, 0.0284, 0.4680, 0.7632,
         0.0581, 0.8266],
        [0.6737, 0.4591, 0.4715, 0.9383, 0.7939, 0.6157, 0.3543, 0.7704, 0.9629,
         0.5136, 0.0828, 0.4886, 0.9063, 0.4598, 0.3493, 0.4506, 0.8380, 0.7756,
         0.9789, 0.3195],
        [0.4387, 0.6109, 0.5523, 0.8139, 0.6977, 0.2085, 0.6233, 0.6960, 0.1639,
         0.2574, 0.5746, 0.4886, 0.8895, 0.3151, 0.5163, 0.0313, 0.8142, 0.1018,
         0.3119, 0.6645],
        [0.1899, 0.2027, 0.4162, 0.5872, 0.2695, 0.1235, 0.5103, 0.6021, 0.5237,
         0.1683, 0.6145, 0.3707, 0.0146, 0.5486, 0.5922, 0.3614, 0.6250, 0.7307,
         0.6773, 0.5773],
        [0.8040, 0.1330, 0.8748, 0.6339, 0.6372, 0.9170, 0.7261, 0.1796, 0.9075,
         0.7318, 0.0779, 0.2887, 0.7177, 0.8936, 0.6263, 0.7646, 0.4454, 0.9667,
         0.4741, 0.8433]])
torch.Size([6, 20])
==============================
tensor([0.7733, 0.1794, 0.7046, 0.2685, 0.5017, 0.4303, 0.3463, 0.7707, 0.0760,
        0.2529, 0.8447, 0.8022, 0.6737, 0.0222, 0.1824, 0.4169, 0.2391, 0.5440,
        0.2884, 0.8852, 0.5118, 0.7946, 0.8106, 0.9128, 0.0094, 0.4233, 0.4691,
        0.6234, 0.2335, 0.9982, 0.3506, 0.7898, 0.9798, 0.9961, 0.5045, 0.0284,
        0.4680, 0.7632, 0.0581, 0.8266, 0.6737, 0.4591, 0.4715, 0.9383, 0.7939,
        0.6157, 0.3543, 0.7704, 0.9629, 0.5136, 0.0828, 0.4886, 0.9063, 0.4598,
        0.3493, 0.4506, 0.8380, 0.7756, 0.9789, 0.3195, 0.4387, 0.6109, 0.5523,
        0.8139, 0.6977, 0.2085, 0.6233, 0.6960, 0.1639, 0.2574, 0.5746, 0.4886,
        0.8895, 0.3151, 0.5163, 0.0313, 0.8142, 0.1018, 0.3119, 0.6645, 0.1899,
        0.2027, 0.4162, 0.5872, 0.2695, 0.1235, 0.5103, 0.6021, 0.5237, 0.1683,
        0.6145, 0.3707, 0.0146, 0.5486, 0.5922, 0.3614, 0.6250, 0.7307, 0.6773,
        0.5773, 0.8040, 0.1330, 0.8748, 0.6339, 0.6372, 0.9170, 0.7261, 0.1796,
        0.9075, 0.7318, 0.0779, 0.2887, 0.7177, 0.8936, 0.6263, 0.7646, 0.4454,
        0.9667, 0.4741, 0.8433])
torch.Size([120])
==============================
tensor([0.7733, 0.1794, 0.7046, 0.2685, 0.5017, 0.4303, 0.3463, 0.7707, 0.0760,
        0.2529, 0.8447, 0.8022, 0.6737, 0.0222, 0.1824, 0.4169, 0.2391, 0.5440,
        0.2884, 0.8852, 0.5118, 0.7946, 0.8106, 0.9128, 0.0094, 0.4233, 0.4691,
        0.6234, 0.2335, 0.9982, 0.3506, 0.7898, 0.9798, 0.9961, 0.5045, 0.0284,
        0.4680, 0.7632, 0.0581, 0.8266, 0.6737, 0.4591, 0.4715, 0.9383, 0.7939,
        0.6157, 0.3543, 0.7704, 0.9629, 0.5136, 0.0828, 0.4886, 0.9063, 0.4598,
        0.3493, 0.4506, 0.8380, 0.7756, 0.9789, 0.3195, 0.4387, 0.6109, 0.5523,
        0.8139, 0.6977, 0.2085, 0.6233, 0.6960, 0.1639, 0.2574, 0.5746, 0.4886,
        0.8895, 0.3151, 0.5163, 0.0313, 0.8142, 0.1018, 0.3119, 0.6645, 0.1899,
        0.2027, 0.4162, 0.5872, 0.2695, 0.1235, 0.5103, 0.6021, 0.5237, 0.1683,
        0.6145, 0.3707, 0.0146, 0.5486, 0.5922, 0.3614, 0.6250, 0.7307, 0.6773,
        0.5773, 0.8040, 0.1330, 0.8748, 0.6339, 0.6372, 0.9170, 0.7261, 0.1796,
        0.9075, 0.7318, 0.0779, 0.2887, 0.7177, 0.8936, 0.6263, 0.7646, 0.4454,
        0.9667, 0.4741, 0.8433])
torch.Size([120])

torch.squeeze和torch.unsqueeze一个是去掉轴，一个是添加轴，可以简单理解为降维度和升维度

print(a.shape)
b = torch.squeeze(a)
print('            ⬇')
b.shape

torch.Size([1, 2, 3, 4, 5])
            ⬇





torch.Size([2, 3, 4, 5])

print(b.shape)
c = torch.unsqueeze(b,0)
print('            ⬆')
c.shape

torch.Size([2, 3, 4, 5])
            ⬆





torch.Size([1, 2, 3, 4, 5])

1.10 类似于矩阵的转置

torch.t和torch.transpose应用于2维矩阵的转置

a_2dim = torch.tensor([[0,1,2],[3,4,5]])
print(a_2dim)
print(torch.t(a_2dim))
print(a_2dim)
print(torch.transpose(a_2dim,1,0))
print(a_2dim)
# 说明转置并不改变原来的tensor

tensor([[0, 1, 2],
        [3, 4, 5]])
tensor([[0, 3],
        [1, 4],
        [2, 5]])
tensor([[0, 1, 2],
        [3, 4, 5]])
tensor([[0, 3],
        [1, 4],
        [2, 5]])
tensor([[0, 1, 2],
        [3, 4, 5]])

高纬度的类转置，用permute()

a = torch.rand((1,2,3,4,5))
print(a.shape)
print('\'转置\'过后\n')
b = a.permute(4,3,2,1,0)
print(b.shape)

torch.Size([1, 2, 3, 4, 5])
'转置'过后

torch.Size([5, 4, 3, 2, 1])

Pytorch也提供了torch.cat和torch.stack用于拼接矩阵，不同的是，torch.cat在已经有的轴dim上拼接矩阵，给定轴的维度可以不同，而其他轴的维度必须相同；torch.stack在新的轴上拼接，他要求被拼接的矩阵所有维度都相同。

a = torch.randn(2,3)
print(a)
b = torch.randn(3,3)
print(b)
# 默认维度为dim=0


print('这是torch.cat方法\n')
c = torch.cat((a,b))
d = torch.cat((b,b,b),dim=1)
print(c)
print(d)
print('='*30)
print(c.shape)
print(d.shape)
print('这是torch.stack方法\n')
c = torch.stack((b,b),dim=1)
d = torch.stack((b,b),dim=0)
print(c)
print(d)
print('='*30)
print(c.shape)
print(d.shape)

tensor([[-0.0383, -0.4312,  1.5002],
        [ 1.5575,  0.8933,  0.8324]])
tensor([[ 0.5628,  0.4439,  2.5350],
        [-0.3516, -0.9461, -0.0221],
        [ 0.2317,  0.2302,  0.1551]])
这是torch.cat方法

tensor([[-0.0383, -0.4312,  1.5002],
        [ 1.5575,  0.8933,  0.8324],
        [ 0.5628,  0.4439,  2.5350],
        [-0.3516, -0.9461, -0.0221],
        [ 0.2317,  0.2302,  0.1551]])
tensor([[ 0.5628,  0.4439,  2.5350,  0.5628,  0.4439,  2.5350,  0.5628,  0.4439,
          2.5350],
        [-0.3516, -0.9461, -0.0221, -0.3516, -0.9461, -0.0221, -0.3516, -0.9461,
         -0.0221],
        [ 0.2317,  0.2302,  0.1551,  0.2317,  0.2302,  0.1551,  0.2317,  0.2302,
          0.1551]])
==============================
torch.Size([5, 3])
torch.Size([3, 9])
这是torch.stack方法

tensor([[[ 0.5628,  0.4439,  2.5350],
         [ 0.5628,  0.4439,  2.5350]],

        [[-0.3516, -0.9461, -0.0221],
         [-0.3516, -0.9461, -0.0221]],

        [[ 0.2317,  0.2302,  0.1551],
         [ 0.2317,  0.2302,  0.1551]]])
tensor([[[ 0.5628,  0.4439,  2.5350],
         [-0.3516, -0.9461, -0.0221],
         [ 0.2317,  0.2302,  0.1551]],

        [[ 0.5628,  0.4439,  2.5350],
         [-0.3516, -0.9461, -0.0221],
         [ 0.2317,  0.2302,  0.1551]]])
==============================
torch.Size([3, 2, 3])
torch.Size([2, 3, 3])

pytorch也提供了torch.split和torch.chunk用于拆分矩阵，不同之处在于前者传入的是每个矩阵的大小，可以传入list，也可以传入整数，而torch.chunk传入的是拆分的矩阵个数。

a = torch.randn(10,3)
for x in torch.split(a,[1,2,3,4],dim=0):
    print(x.shape)
print('='*60)    
for x in torch.split(a,4,dim=0):
    print(x.shape)
print('='*60)
for x in torch.chunk(a,4,dim=0):
    print(x.shape)

torch.Size([1, 3])
torch.Size([2, 3])
torch.Size([3, 3])
torch.Size([4, 3])
============================================================
torch.Size([4, 3])
torch.Size([4, 3])
torch.Size([2, 3])
============================================================
torch.Size([3, 3])
torch.Size([3, 3])
torch.Size([3, 3])
torch.Size([1, 3])

1.11 pytorch的reduction操作

Recucition运算的特点是它往往对一个Tensor内的元素做归约操作，比如torch.max和torch.min等，以及还提供了dim=？？参数，直接作用哪一个维度

import torch
import numpy as np
# 默认求取全局最大值
a = torch.tensor([[1,2,3],[3,4,1]])
print(a)
print('全局最大值：',torch.max(a))
# 指定维度，返回最大值及其索引
print(torch.max(a,dim=0)) # 行 之间比较，竖着看

print('沿着横轴计算每一行的累乘：',torch.cumsum(a,dim=0)) # 行 之间累加，竖着看

print('沿着纵轴计算每一行的累乘：',torch.cumprod(a,dim=1))# 列 之间累成，竖着看

# 计算矩阵的均值、中位数、协方差
a = torch.Tensor([[1,2,3],[3,4,1]]) # 注意这里只能用torch.Tensor来创建张量，不然会报错
print('均值：',a.mean(),'中位数：',a.median(),'协方差',a.std())

tensor([[1, 2, 3],
        [3, 4, 1]])
全局最大值： tensor(4)
torch.return_types.max(
values=tensor([3, 4, 3]),
indices=tensor([1, 1, 0]))
沿着横轴计算每一行的累乘： tensor([[1, 2, 3],
        [4, 6, 4]])
沿着纵轴计算每一行的累乘： tensor([[ 1,  2,  6],
        [ 3, 12, 12]])
均值： tensor(2.3333) 中位数： tensor(2.) 协方差 tensor(1.2111)

# torch.unique()找出矩阵中独一无二的元素，类似于集合的功能
a = torch.randint(0,3,(3,3))
print(a)
print('-'*60)
print("矩阵中的出现{}元素".format(torch.unique(a)))

tensor([[1, 2, 1],
        [1, 1, 2],
        [2, 0, 2]])
------------------------------------------------------------
矩阵中的出现tensor([0, 1, 2])元素