Tensors
Tensors
是一个特殊的数据结构,非常类似于数组和矩阵,在 PyTorch
中,我们使用 tensors
编码模型的输入和输出,以及模型的参数。
Tensors
非常类似于 NumPy
的 ndarrays
, tensors
可以运行在 GPU 以及其他硬件加速器上,tensors
还可以与 NumPy
还可以共享底层内存,消除复制数据的需要(见 Bridge with NumPy ) ,Tensors 也为自动微分进行了优化 ( 见Autograd )
如果熟悉 ndarrays
,可以直接看 Tensor
的 API,
否则继续。
Initializing a Tensor ( 初始化 Tensor
)
直接从数据
Tensors 可以直接从数据创建,数据的类型自动推理到输入中。
import numpy as np
import torch
data = [[1, 2], [3, 4]]
x_data = torch.tensor(data)
x_data
tensor([[1, 2],
[3, 4]])
从 NumPy 数据
Tensors 可以从 NumPy 的 array 创建 ( Bridge with NumPy )
np_array = np.array(data)
x_np = torch.from_numpy(np_array)
x_np
tensor([[1, 2],
[3, 4]], dtype=torch.int32)
从其他 Tensor
新的 Tensor 保留了参数 Tensor 的属性(形状,数据类型),除非显式重写
# 保留了数据类型和形状
x_ones = torch.ones_like(x_data)
print(f"Ones Tensor: \n{x_ones}")
# 显式重写 数据类型
x_rand = torch.rand_like(x_data, dtype=torch.float)
print(f"Random Tensor: \n{x_rand}")
Ones Tensor:
tensor([[1, 1],
[1, 1]])
Random Tensor:
tensor([[0.1812, 0.3162],
[0.9739, 0.1262]])
torch.int64
根据形状创建随机值和常量值的 Tensor
shape = (2, 3, )
rand_tensor = torch.rand(shape)
ones_tensor = torch.ones(shape)
zeros_tensor = torch.zeros(shape)
print(f"Random Tensor: \n {rand_tensor}")
print(f"Ones Tensor: \n {ones_tensor}")
print(f"Zeros Tensor: \n {zeros_tensor}")
Random Tensor:
tensor([[0.9553, 0.4883, 0.5537],
[0.4053, 0.8551, 0.1555]])
Ones Tensor:
tensor([[1., 1., 1.],
[1., 1., 1.]])
Zeros Tensor:
tensor([[0., 0., 0.],
[0., 0., 0.]])
rand
与randn
区别是,
rand
生成[0,1]
之间均匀分布的随机数填充tensor
randn
生成 均值为0, 方差为 1 的正态分布随机数填充tensor
Attributes of a Tensor ( Tensor 的属性 )
Tensor 属性描述了
shape
,datatype
和device
t = torch.rand(3, 4)
print(f"Shape of t: {t.shape}")
print(f"Datatype of t: {t.dtype}")
print(f"Device of t: {t.device}")
Shape of t: torch.Size([3, 4])
Datatype of t: torch.float32
Device of t: cpu
Operations on Tensors ( Tensor 操作)
超过 100 种 tensor 操作,包括算术、线性代数、矩阵操作(转置、索引、切片),采样等等,更多的描述请看 here
每个操作都可以运行在 GPU
上,默认情况下,运行在 CPU
上,需要显式将 tensor
转移到 GPU
上 (确定 GPU
可用的情况下,利用 .to
方法)。记住:大的 tensor
在不同的 device
上 非常耗时和占用内存。
索引和切片
t = torch.rand(4, 4)
# tensor([[0.7647, 0.7959, 0.6392, 0.4798],
# [0.1062, 0.0885, 0.5527, 0.3050],
# [0.5985, 0.7518, 0.1282, 0.1422],
# [0.1410, 0.6520, 0.1167, 0.8047]])
print(f"First row: {t[0]}")
print(f"First col: {t[:, 0]}")
print(f"Last col: {t[..., -1]}") # print(f"Last col: {t[:, -1]}")
t[:,1] = 0
print(t)
First row: tensor([0.7647, 0.7959, 0.6392, 0.4798])
First col: tensor([0.7647, 0.1062, 0.5985, 0.1410])
Last col: tensor([0.4798, 0.3050, 0.1422, 0.8047])
tensor([[0.7647, 0.0000, 0.6392, 0.4798],
[0.1062, 0.0000, 0.5527, 0.3050],
[0.5985, 0.0000, 0.1282, 0.1422],
[0.1410, 0.0000, 0.1167, 0.8047]])
join
You can use torch.cat to concatenate a sequence of tensors along a given dimension. See also torch.stack, another tensor joining op that is subtly different from torch.cat.
你可以使用 torch.cat
在指定维度上连接 tensors
,可以看 torch.stack, 与 cat
略有不同的连接。
t1 = torch.cat([t, t, t], dim=1)
print(t1)
tensor([[0.7647, 0.0000, 0.6392, 0.4798, 0.7647, 0.0000, 0.6392, 0.4798, 0.7647,
0.0000, 0.6392, 0.4798],
[0.1062, 0.0000, 0.5527, 0.3050, 0.1062, 0.0000, 0.5527, 0.3050, 0.1062,
0.0000, 0.5527, 0.3050],
[0.5985, 0.0000, 0.1282, 0.1422, 0.5985, 0.0000, 0.1282, 0.1422, 0.5985,
0.0000, 0.1282, 0.1422],
[0.1410, 0.0000, 0.1167, 0.8047, 0.1410, 0.0000, 0.1167, 0.8047, 0.1410,
0.0000, 0.1167, 0.8047]]) torch.Size([4, 12])
t2 = torch.cat([t, t, t], dim=0)
print(t2)
tensor([[0.7647, 0.0000, 0.6392, 0.4798],
[0.1062, 0.0000, 0.5527, 0.3050],
[0.5985, 0.0000, 0.1282, 0.1422],
[0.1410, 0.0000, 0.1167, 0.8047],
[0.7647, 0.0000, 0.6392, 0.4798],
[0.1062, 0.0000, 0.5527, 0.3050],
[0.5985, 0.0000, 0.1282, 0.1422],
[0.1410, 0.0000, 0.1167, 0.8047],
[0.7647, 0.0000, 0.6392, 0.4798],
[0.1062, 0.0000, 0.5527, 0.3050],
[0.5985, 0.0000, 0.1282, 0.1422],
[0.1410, 0.0000, 0.1167, 0.8047]]) torch.Size([12, 4])
Arithmetic operations (算术运算)
tensor = torch.rand(4, 4)
y1 = tensor @ tensor.T
y2 = tensor.matmul(tensor.t())
y3 = torch.rand_like(tensor)
torch.matmul(tensor, tensor.T, out=y3)
print("y1=", y1)
print("y2=", y2)
print("y3=", y3)
###########
z1 = tensor * tensor
z2 = tensor.mul(tensor)
z3 = torch.rand_like(tensor)
torch.mul(tensor, tensor, out=z3)
print("z1=", z1)
print("z2=", z2)
print("z3=", z3)
y1= tensor([[1.8622, 0.7500, 1.3220, 1.5922],
[0.7500, 0.4831, 0.4377, 0.7000],
[1.3220, 0.4377, 1.6632, 1.4770],
[1.5922, 0.7000, 1.4770, 1.9611]])
y2= tensor([[1.8622, 0.7500, 1.3220, 1.5922],
[0.7500, 0.4831, 0.4377, 0.7000],
[1.3220, 0.4377, 1.6632, 1.4770],
[1.5922, 0.7000, 1.4770, 1.9611]])
y3= tensor([[1.8622, 0.7500, 1.3220, 1.5922],
[0.7500, 0.4831, 0.4377, 0.7000],
[1.3220, 0.4377, 1.6632, 1.4770],
[1.5922, 0.7000, 1.4770, 1.9611]])
z1= tensor([[4.8304e-01, 7.3252e-01, 8.0845e-02, 5.6575e-01],
[1.1725e-03, 8.6222e-02, 3.8380e-05, 3.9572e-01],
[6.8993e-01, 6.5055e-03, 6.0031e-01, 3.6649e-01],
[7.5780e-02, 3.9919e-01, 8.5609e-01, 6.3000e-01]])
z2= tensor([[4.8304e-01, 7.3252e-01, 8.0845e-02, 5.6575e-01],
[1.1725e-03, 8.6222e-02, 3.8380e-05, 3.9572e-01],
[6.8993e-01, 6.5055e-03, 6.0031e-01, 3.6649e-01],
[7.5780e-02, 3.9919e-01, 8.5609e-01, 6.3000e-01]])
z3= tensor([[4.8304e-01, 7.3252e-01, 8.0845e-02, 5.6575e-01],
[1.1725e-03, 8.6222e-02, 3.8380e-05, 3.9572e-01],
[6.8993e-01, 6.5055e-03, 6.0031e-01, 3.6649e-01],
[7.5780e-02, 3.9919e-01, 8.5609e-01, 6.3000e-01]])
mul
与matmul
的区别是:mul
是元素级乘法,matmul
是矩阵乘法。
单元素的tensor
例如将 tensor
里所有的值聚合成一个值,可以利用 item()
将它转换为 Python
的数值数据。
agg = tensor.sum()
agg_item = agg.item()
print(agg_item, type(agg_item))
8.468034744262695 <class 'float'>
In-place operations
会将操作计算的值内部存储。可以用带 _
下划线的方法,例如 x.copy_(y)
和 x.t_()
会改变 x
的值
print(f"{tensor} \n")
tensor.add_(5)
print(tensor)
tensor([[0.6950, 0.8559, 0.2843, 0.7522],
[0.0342, 0.2936, 0.0062, 0.6291],
[0.8306, 0.0807, 0.7748, 0.6054],
[0.2753, 0.6318, 0.9253, 0.7937]])
tensor([[5.6950, 5.8559, 5.2843, 5.7522],
[5.0342, 5.2936, 5.0062, 5.6291],
[5.8306, 5.0807, 5.7748, 5.6054],
[5.2753, 5.6318, 5.9253, 5.7937]])
in-place 操作会节省内存,但在计算导数时可能会出现问题,因为会立即丢失历史记录。因此,不鼓励使用它们。
Bridge with NumPy
Tensor to NumPy array
t = torch.ones(5)
print(f"t: {t} \n")
n = t.numpy()
print(f"n: {n} \n")
t: tensor([1., 1., 1., 1., 1.])
n: [1. 1. 1. 1. 1.]
tensor 的变化也改变了 NumPy 数组。
t.add_(3)
print(f"t: {t} \n")
print(f"n: {n} \n")
t: tensor([4., 4., 4., 4., 4.])
n: [4. 4. 4. 4. 4.]
NumPy array to Tensor
n = np.ones(3)
t = torch.from_numpy(n)
np.add(n, 1, out=n)
print(f"t: {t}")
print(f"n: {n}")
t: tensor([2., 2., 2.], dtype=torch.float64)
n: [2. 2. 2.]
NumPy数组的变化也会影响 tensor