PyTorch官方Tutorials Tenosrs

PyTorch官方Tutorials

跟着PyTorch官方Tutorials码的，便于理解自己稍有改动代码并添加注释，IDE用的jupyter notebook

链接: tutorials-Tensors

TENSORS

Tensors are a specialized data structure that are very similar to arrays and matrices.
In PyTorch, we use tensors to encode the inputs and outputs of a model, as well as the model’s parameters.

tensor是一种与数组和矩阵非常类似的数据结构
在pytorch里 用tensor来对一个模型中的输入和输出以及模型的参数进行编码

tensors are similar to NumPy’s ndarrays, except that tensors can run on GPUs or other hardware accelerators.
In fact, tensors and NumPy arrays can often share the same underlying memory, eliminating the need to copy data (see Bridge with NumPy).

tensor和numpy中的ndarray非常像,但是tensor可以在GPU或者其他硬件加速器上运行
事实上,tensor和numpy的array经常共用潜在内存,这样就免除了复制数据的麻烦

Tensors are also optimized for automatic differentiation (we’ll see more about that later in the Autograd section).
If you’re familiar with ndarrays, you’ll be right at home with the Tensor API. If not, follow along!

tensor针对自动微分做了优化

import torch
import numpy as np

Initializing a Tensor

Tensors can be initialized in various ways. Take a look at the following examples:

tensor有多种初始化方式

Directly from data

Tensors can be created directly from data. The data type is automatically inferred.

直接使用数据初始化 类型自动关联

data=[[1,2],[3,4]]
x_data=torch.tensor(data)
x_data

tensor([[1, 2],
        [3, 4]])

x_data_tensor=torch.tensor([[1,2],[3,4]])
x_data_tensor

tensor([[1, 2],
        [3, 4]])

From a NumPy array

Tensors can be created from NumPy arrays (and vice versa - see Bridge with NumPy).

tensor也可以由numpy中的array初始化

np_array=np.array(data)
x_np=torch.from_numpy(np_array)
x_np

tensor([[1, 2],
        [3, 4]], dtype=torch.int32)

np_array_direct=np.array([[1,2],[3,4]])
np_array_direct

array([[1, 2],
       [3, 4]])

From another tensor:

The new tensor retains the properties (shape, datatype) of the argument tensor, unless explicitly overridden.

用另一个tensor初始化 保留形状 数字类型等特征 除非特意改写

x_ones=torch.ones_like(x_data)
print(f'Ones Tensor:\n {x_ones}\n')

Ones Tensor:
 tensor([[1, 1],
        [1, 1]])

#注意此处要有dtype=torch.float

#rand_like(shape) ones_like(shape) zeros_like(shape)
#shape为矩阵类型
x_rand=torch.rand_like(x_data,dtype=torch.float)
print(f'Random Tensor:\n {x_rand}\n')

Random Tensor:
 tensor([[0.2766, 0.0389],
        [0.1741, 0.9451]])

With random or constant values:

shape is a tuple of tensor dimensions. In the functions below, it determines the dimensionality of the output tensor.

shape是一个表示tensor维度的元组

#torch.rand(shape) torch.ones(shape) torch.zeros(shape)
#shape为元组类型
shape=(2,3)
rand_tensor=torch.rand(shape)
ones_tensor=torch.ones(shape)
zeros_tensor=torch.zeros(shape)

print(f"Random Tensor:\n {rand_tensor}\n")
print(f"Ones Tensor:\n {ones_tensor}\n")
print(f"Zeros Tensor:\n {zeros_tensor}\n")

Random Tensor:
 tensor([[0.6058, 0.4060, 0.1762],
        [0.8425, 0.0200, 0.2036]])

Ones Tensor:
 tensor([[1., 1., 1.],
        [1., 1., 1.]])

Zeros Tensor:
 tensor([[0., 0., 0.],
        [0., 0., 0.]])

Attributes of a Tensor

Tensor attributes describe their shape, datatype, and the device on which they are stored.

tensor的参数描述了shape datatype 以及存储的device 即shape dtype device三个函数

#torch.rand生成的是张量
tensor=torch.rand(3,4)
tensor

tensor([[0.5195, 0.5672, 0.4530, 0.6773],
        [0.9247, 0.5316, 0.3457, 0.7984],
        [0.4321, 0.7133, 0.5241, 0.0926]])

#这里不用加括号
print(f"Shape of tensor: {tensor.shape}")
print(f"Datatype of tensor: {tensor.dtype}")
print(f"Device tensor is stored on: {tensor.device}")

Shape of tensor: torch.Size([3, 4])
Datatype of tensor: torch.float32
Device tensor is stored on: cpu

Operations on Tensors

Over 100 tensor operations, including arithmetic, linear algebra, matrix manipulation (transposing, indexing, slicing), sampling and more are comprehensively described here.

tensor支持包括基本算数 线性代数 矩阵乘法等超过100种运算

Each of these operations can be run on the GPU (at typically higher speeds than on a CPU). If you’re using Colab, allocate a GPU by going to Runtime > Change runtime type > GPU.

每种操作都支持在gpu上运行 一般都比在cpu上快

By default, tensors are created on the CPU. We need to explicitly move tensors to the GPU using .to method (after checking for GPU availability). Keep in mind that copying large tensors across devices can be expensive in terms of time and memory!

tensor默认创建在cpu上 我们需要专门将它移到gpu上 具体在检验gpu可用后使用.to方法

移动大tensor时开销可能会比较大

#把tensor移动到gpu
if torch.cuda.is_available():
    tensor=tensor.to('cuda')

print(f'Tensor\'s device: {tensor.device}')

Tensor's device: cuda:0

Try out some of the operations from the list. If you’re familiar with the NumPy API, you’ll find the Tensor API a breeze to use.

Standard numpy-like indexing and slicing:

tensor可以像numpy一样支持索引和切片操作

tensor=torch.randn(4,4)
print(tensor)

print(f'First row   : {tensor[0]}')
#print(f'First row {tensor[0,:]}')
print(f'First column: {tensor[:,0]}')
print(f'Last column : {tensor[:,-1]}')
print(f'Last row    : {tensor[-1,:]}')

tensor([[ 1.2115e-02, -4.7674e-04, -1.6909e+00,  9.5488e-02],
        [ 6.5391e-01,  6.2818e-01,  1.8119e-01,  8.8987e-01],
        [ 8.4265e-01, -2.2776e+00,  1.9617e-01,  3.3608e-01],
        [ 1.2253e+00,  1.7447e-01,  5.4989e-01,  2.2819e-01]])
First row   : tensor([ 1.2115e-02, -4.7674e-04, -1.6909e+00,  9.5488e-02])
First column: tensor([0.0121, 0.6539, 0.8427, 1.2253])
Last column : tensor([0.0955, 0.8899, 0.3361, 0.2282])
Last row    : tensor([1.2253, 0.1745, 0.5499, 0.2282])

tensor[:,1]=0
print(tensor)

tensor([[ 0.0121,  0.0000, -1.6909,  0.0955],
        [ 0.6539,  0.0000,  0.1812,  0.8899],
        [ 0.8427,  0.0000,  0.1962,  0.3361],
        [ 1.2253,  0.0000,  0.5499,  0.2282]])

Joining tensors

You can use torch.cat to concatenate a sequence of tensors along a given dimension. See also torch.stack, another tensor joining op that is subtly different from torch.cat.

使用torch.cat来将tensor在某一维度进行连接合并 torch.satck也有此种功能

#dim=0按行合并,dim=1按列合并 按谁合并谁不变
t1=torch.cat([tensor,tensor,tensor],dim=1)
print(t1)

t2=torch.cat([tensor,tensor,tensor],dim=0)
print(t2)

---------------------------------------------------------------------------

NameError                                 Traceback (most recent call last)

<ipython-input-11-d2d62d4aaf7b> in <module>
      1 #dim=0按行合并,dim=1按列合并 按谁合并谁不变
----> 2 t1=torch.cat([tensor,tensor,tensor],dim=1)
      3 print(t1)
      4 
      5 t2=torch.cat([tensor,tensor,tensor],dim=0)


NameError: name 'tensor' is not defined

Arithmetic operations

算数操作 @ .matmul是矩阵乘法 * .mul是矩阵元素对应相乘

# This computes the matrix multiplication between two tensors. 
# y1, y2, y3 will have the same value

print('tensor:\n',tensor)

#区分tensor.matmul和torch.matmul
y1=tensor@tensor.T
y2=tensor.matmul(tensor.T)

y3=torch.rand_like(tensor)
torch.matmul(tensor,tensor.T,out=y3)

print('y1:\n',y1)
print('y2:\n',y2)
print('y3:\n',y3)

# This computes the element-wise product. z1, z2, z3 will have the same value
z1=tensor*tensor
z2=tensor.mul(tensor)

z3=torch.rand_like(tensor)
torch.mul(tensor,tensor,out=z3)

print('z1:\n',z1)
print('z2:\n',z2)
print('z3:\n',z3)

tensor:
 tensor([[ 0.0121,  0.0000, -1.6909,  0.0955],
        [ 0.6539,  0.0000,  0.1812,  0.8899],
        [ 0.8427,  0.0000,  0.1962,  0.3361],
        [ 1.2253,  0.0000,  0.5499,  0.2282]])
y1:
 tensor([[ 2.8686, -0.2135, -0.2894, -0.8932],
        [-0.2135,  1.2523,  0.8856,  1.1039],
        [-0.2894,  0.8856,  0.8615,  1.2170],
        [-0.8932,  1.1039,  1.2170,  1.8557]])
y2:
 tensor([[ 2.8686, -0.2135, -0.2894, -0.8932],
        [-0.2135,  1.2523,  0.8856,  1.1039],
        [-0.2894,  0.8856,  0.8615,  1.2170],
        [-0.8932,  1.1039,  1.2170,  1.8557]])
y3:
 tensor([[ 2.8686, -0.2135, -0.2894, -0.8932],
        [-0.2135,  1.2523,  0.8856,  1.1039],
        [-0.2894,  0.8856,  0.8615,  1.2170],
        [-0.8932,  1.1039,  1.2170,  1.8557]])
z1:
 tensor([[1.4677e-04, 0.0000e+00, 2.8593e+00, 9.1180e-03],
        [4.2760e-01, 0.0000e+00, 3.2830e-02, 7.9187e-01],
        [7.1007e-01, 0.0000e+00, 3.8482e-02, 1.1295e-01],
        [1.5013e+00, 0.0000e+00, 3.0238e-01, 5.2072e-02]])
z2:
 tensor([[1.4677e-04, 0.0000e+00, 2.8593e+00, 9.1180e-03],
        [4.2760e-01, 0.0000e+00, 3.2830e-02, 7.9187e-01],
        [7.1007e-01, 0.0000e+00, 3.8482e-02, 1.1295e-01],
        [1.5013e+00, 0.0000e+00, 3.0238e-01, 5.2072e-02]])
z3:
 tensor([[1.4677e-04, 0.0000e+00, 2.8593e+00, 9.1180e-03],
        [4.2760e-01, 0.0000e+00, 3.2830e-02, 7.9187e-01],
        [7.1007e-01, 0.0000e+00, 3.8482e-02, 1.1295e-01],
        [1.5013e+00, 0.0000e+00, 3.0238e-01, 5.2072e-02]])

# matrix1*matrix2 相同位置元素相乘
m1=torch.randn(2,3)
print('m1:\n',m1)

m2=torch.randn(2,3)
print('m2:\n',m2)

m3=m1*m2
print('m3:\n',m3)

for i in range(2):
    for j in range(3):
        print(m1[i][j]*m2[i][j],end='')
    print()

m1:
 tensor([[-0.7677,  1.7526,  1.6339],
        [ 0.0568,  1.1335,  0.6162]])
m2:
 tensor([[-0.2034,  2.9472,  0.9468],
        [-1.3366, -0.3348,  0.2394]])
m3:
 tensor([[ 0.1562,  5.1652,  1.5470],
        [-0.0759, -0.3795,  0.1475]])
tensor(0.1562)tensor(5.1652)tensor(1.5470)
tensor(-0.0759)tensor(-0.3795)tensor(0.1475)

Single-element tensors

If you have a one-element tensor, for example by aggregating all values of a tensor into one value, you can convert it to a Python numerical value using item():

单元素tensor使用item()转为数值 .sum .sum(axis=0)加成一行 .sum(axis=1)加成一列 加成谁谁不变

tensor=torch.randn(2,3)
tensor

tensor([[ 0.4111, -0.5057, -1.3144],
        [ 0.2235,  0.7176,  2.4131]])

# 所有元素相加
agg=tensor.sum()
print('agg',agg)
print('agg.item()',agg.item(),'dtype',agg.dtype)

agg tensor(1.9453)
agg.item() 1.9453327655792236 dtype torch.float32

# 加成一行
agg=tensor.sum(axis=0)
agg

tensor([0.6346, 0.2119, 1.0988])

# 加成一列,变成行
agg=tensor.sum(axis=1)
agg

tensor([-1.4089,  3.3543])

In-place operations

Operations that store the result into the operand are called in-place. They are denoted by a _ suffix. For example: x.copy_(y), x.t_(), will change x.

in-place操作 _后缀表示改变当前变量的值

print(tensor,'\n')

#每个元素+5,广播机制
tensor.add_(5)
print(tensor)

tensor([[ 0.4111, -0.5057, -1.3144],
        [ 0.2235,  0.7176,  2.4131]]) 

tensor([[5.4111, 4.4943, 3.6856],
        [5.2235, 5.7176, 7.4131]])

In-place operations save some memory, but can be problematic when computing derivatives because of an immediate loss of history. Hence, their use is discouraged.

节约内存 但是不利于微分计算 因为会丢失历史 不推荐使用

Bridge with NumPy

Tensors on the CPU and NumPy arrays can share their underlying memory locations, and changing one will change the other.

cpu上的tensor和numpy array共用内存

Tensor to NumPy array

t=torch.ones(5)
t

tensor([1., 1., 1., 1., 1.])

n=t.numpy()
n

array([1., 1., 1., 1., 1.], dtype=float32)

A change in the tensor reflects in the NumPy array.

#注意这里的t.add_()不是t.add()
t.add_(1)
print(n)

[2. 2. 2. 2. 2.]

NumPy array to Tensor

n=np.ones(5)
n

array([1., 1., 1., 1., 1.])

t=torch.from_numpy(n)
t

tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

Changes in the NumPy array reflects in the tensor.

#注意numpy的加法 np.add(n1,x,out=n1)
np.add(n,1,out=n)
t

tensor([2., 2., 2., 2., 2.], dtype=torch.float64)