pytorch学习 基本组成、基本练习

1. 基本结构

pytorch主要分为以下几个模块来训练模型：

1. tensor:tensor为基本结构，可以直接创建，从list创建以及由numpy数组得到，torch还提供一套运算以及shape变换方式。
2. Variable：自动求导机制，利用Variable包装tensor后，便可以使用其求导的功能了，有点像个装饰器。
3. nn:nn模块是整个pytorch的核心，自己设计的Net(),继承nn.Model后可以提取模型参数，进行前向forward()运算（自己设计），以及后向运算（自动），nn提供基本网络结构单元，例如nn.Linear(),nn.Conv2d()等，还提供基本损失函数nn.CrossEntropyLoss等。
4. torch.optim：该模块提供自动求导更新参数等功能，用它封装模型参数nn.parameter()后，loss求导后，可以用.step来更新整个参数。
5. torch.utils.data.DataSet:该模块提供加载数据初始化的方式，完善好getitem和len的接口后，便可以利用DataLoader多进程加载数据。

2.基本练习

1.Tensor学习

import torch

1.1 tensorf创建

1.1 list 转 tensor

V_data = [1,2,3,4]

V = torch.tensor(V_data)

TypeErrorTraceback (most recent call last)

<ipython-input-4-9481b0be1435> in <module>()
----> 1 V = torch.tensor(V_data)


TypeError: 'module' object is not callable

V = torch.Tensor(V_data)

print V

 1
 2
 3
 4
[torch.FloatTensor of size 4]

1.2 通过tensor自定义创建tensor

1.2.1 torch.rand

?torch.rand

rand创建的是随机均匀分布的，值的范围为0到1

torch.rand(2,3) 

 0.7315  0.7298  0.0671
 0.7685  0.0843  0.5733
[torch.FloatTensor of size 2x3]

1.2.2 torch.randn

?torch.randn

torch.randn为正太分布，均值为0，方差为1

torch.randn(1,2)

-0.7939 -0.7320
[torch.FloatTensor of size 1x2]

1.2 tensor数学运算

1.2.1 加法

torch.Tensor([1,2,3]) + torch.Tensor([4,5,6])

 5
 7
 9
[torch.FloatTensor of size 3]

1.2.2 tensor拼接

维度解读

k0·k1·k2·k3·k4·k5

如上，最左边的为第0维，从左到右依次递增，拼接第几维就是那一维度相加，其余维度不变（也是约束条件）

?torch.cat

torch.cat(torch.Tensor([1,2,3]),torch.Tensor([4,5,6]),0)

TypeErrorTraceback (most recent call last)

<ipython-input-20-367eddc16eb7> in <module>()
----> 1 torch.cat(torch.Tensor([1,2,3]),torch.Tensor([4,5,6]),0)


TypeError: cat received an invalid combination of arguments - got (torch.FloatTensor, torch.FloatTensor, int), but expected one of:
 * (sequence[torch.FloatTensor] seq)
 * (sequence[torch.FloatTensor] seq, int dim)

torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],0)

 1
 2
 3
 4
 5
 6
[torch.FloatTensor of size 6]

torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],1)

RuntimeErrorTraceback (most recent call last)

<ipython-input-23-ffc600f25730> in <module>()
----> 1 torch.cat([torch.Tensor([1,2,3]),torch.Tensor([4,5,6])],1)


RuntimeError: dim out of range - got 1 but the tensor is only 1D

torch.Tensor([1,2,3])

 1
 2
 3
[torch.FloatTensor of size 3]

torch.Tensor([[1,2,3]])

 1  2  3
[torch.FloatTensor of size 1x3]

torch.Tensor([[[1,2,3]]])

(0 ,.,.) = 
  1  2  3
[torch.FloatTensor of size 1x1x3]

torch.cat([torch.Tensor([[1,2,3]]),torch.Tensor([[4,5,6]])],0)

 1  2  3
 4  5  6
[torch.FloatTensor of size 2x3]

torch.cat([torch.Tensor([[1,2,3]]),torch.Tensor([[4,5,6]])],1)

 1  2  3  4  5  6
[torch.FloatTensor of size 1x6]

1.2.3 tensor变形

x = torch.randn(2,3,4)

x

(0 ,.,.) = 
 -0.2756 -0.7231 -0.2633 -0.5982
  1.1090  1.0519  0.7157  0.4336
  1.4142 -0.1507 -0.1062 -1.3992

(1 ,.,.) = 
  0.4155  1.1804 -1.0576 -2.0781
  0.6167 -0.0995 -0.1453  0.2573
 -1.3292 -1.8697 -0.2988 -0.0990
[torch.FloatTensor of size 2x3x4]

x.view(1,12)

RuntimeErrorTraceback (most recent call last)

<ipython-input-33-c96cc26a3a86> in <module>()
----> 1 x.view(1,12)


RuntimeError: invalid argument 2: size '[1 x 12]' is invalid for input of with 24 elements at /pytorch/torch/lib/TH/THStorage.c:41

x.view(12,1)

RuntimeErrorTraceback (most recent call last)

<ipython-input-34-a611982e8650> in <module>()
----> 1 x.view(12,1)


RuntimeError: invalid argument 2: size '[12 x 1]' is invalid for input of with 24 elements at /pytorch/torch/lib/TH/THStorage.c:41

x.view(2,12)

Columns 0 to 9 
-0.2756 -0.7231 -0.2633 -0.5982  1.1090  1.0519  0.7157  0.4336  1.4142 -0.1507
 0.4155  1.1804 -1.0576 -2.0781  0.6167 -0.0995 -0.1453  0.2573 -1.3292 -1.8697

Columns 10 to 11 
-0.1062 -1.3992
-0.2988 -0.0990
[torch.FloatTensor of size 2x12]

x.view(-1)

-0.2756
-0.7231
-0.2633
-0.5982
 1.1090
 1.0519
 0.7157
 0.4336
 1.4142
-0.1507
-0.1062
-1.3992
 0.4155
 1.1804
-1.0576
-2.0781
 0.6167
-0.0995
-0.1453
 0.2573
-1.3292
-1.8697
-0.2988
-0.0990
[torch.FloatTensor of size 24]

x.view(12,-1)

-0.2756 -0.7231
-0.2633 -0.5982
 1.1090  1.0519
 0.7157  0.4336
 1.4142 -0.1507
-0.1062 -1.3992
 0.4155  1.1804
-1.0576 -2.0781
 0.6167 -0.0995
-0.1453  0.2573
-1.3292 -1.8697
-0.2988 -0.0990
[torch.FloatTensor of size 12x2]

1.3 求导机制

import torch.autograd.Variable as Variable

ImportErrorTraceback (most recent call last)

<ipython-input-39-4a31155c778b> in <module>()
----> 1 import torch.autograd.Variable as Variable


ImportError: No module named Variable

import torch.autograd as autograd

x = autograd.Variable(torch.randn(2,3))

y = autograd.Variable(torch.randn(2,3))

x+y

Variable containing:
 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]

z = x+y

z.data

 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]

z

Variable containing:
 0.0876 -0.5458 -0.6579
-0.9776  0.6228  0.7369
[torch.FloatTensor of size 2x3]

z.creator

AttributeErrorTraceback (most recent call last)

<ipython-input-49-a6571e6541ef> in <module>()
----> 1 z.creator


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'

z1 = autograd.Variable(torch.randn(2,3),requires_grad=True)+autograd.Variable(torch.randn(2,3),requires_grad=True)

z1.data

-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]

z1

Variable containing:
-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]

z1.creator

AttributeErrorTraceback (most recent call last)

<ipython-input-53-8fec5dcfb2ec> in <module>()
----> 1 z1.creator


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'

z1.sum()

Variable containing:
-8.0810
[torch.FloatTensor of size 1]

z1.backward

<bound method Variable.backward of Variable containing:
-1.2218 -0.6063 -1.8739
-1.2457 -2.4399 -0.6936
[torch.FloatTensor of size 2x3]
>

z1.backward()

RuntimeErrorTraceback (most recent call last)

<ipython-input-58-19fb924d2b5b> in <module>()
----> 1 z1.backward()


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in backward(self, gradient, retain_graph, create_graph, retain_variables)
    154                 Variable.
    155         """
--> 156         torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
    157 
    158     def register_hook(self, hook):


/usr/lib64/python2.7/site-packages/torch/autograd/__init__.pyc in backward(variables, grad_variables, retain_graph, create_graph, retain_variables)
     84         grad_variables = list(grad_variables)
     85 
---> 86     grad_variables, create_graph = _make_grads(variables, grad_variables, create_graph)
     87 
     88     if retain_variables is not None:


/usr/lib64/python2.7/site-packages/torch/autograd/__init__.pyc in _make_grads(outputs, grads, user_create_graph)
     32             if out.requires_grad:
     33                 if out.numel() != 1:
---> 34                     raise RuntimeError("grad can be implicitly created only for scalar outputs")
     35                 data = out.data
     36                 new_grads.append(


RuntimeError: grad can be implicitly created only for scalar outputs

！！！求导只能用一个scalar outputs，这个怎么翻译？常量？而不是tensor

# Variable 可以从 Tensor 中创建
x = autograd.Variable( torch.Tensor([1., 2., 3]), requires_grad=True )
# 可以使用.data属性来取出 Variable 中的数据
print(x.data)

# Tensor 支持的运算，Variable 都支持
y = autograd.Variable( torch.Tensor([4., 5., 6]), requires_grad=True )
z = x + y
print(z.data)

# 但是 Variable 的特点是，它在运算中可以自动建立父子节点关系
print(z.creator)

 1
 2
 3
[torch.FloatTensor of size 3]


 5
 7
 9
[torch.FloatTensor of size 3]






AttributeErrorTraceback (most recent call last)

<ipython-input-59-d68a9640cdfd> in <module>()
     10 
     11 # 但是 Variable 的特点是，它在运算中可以自动建立父子节点关系
---> 12 print(z.creator)


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'

# 让我们把z中的值都加起来（5+7+9）
s = z.sum()
print(s)
print(s.creator)

Variable containing:
 21
[torch.FloatTensor of size 1]






AttributeErrorTraceback (most recent call last)

<ipython-input-60-df2387da0c35> in <module>()
      2 s = z.sum()
      3 print(s)
----> 4 print(s.creator)


/usr/lib64/python2.7/site-packages/torch/autograd/variable.pyc in __getattr__(self, name)
     63         if name in self._fallthrough_methods:
     64             return getattr(self.data, name)
---> 65         return object.__getattribute__(self, name)
     66 
     67     def __getitem__(self, key):


AttributeError: 'Variable' object has no attribute 'creator'

s.backward() 
print(x.grad)

Variable containing:
 1
 1
 1
[torch.FloatTensor of size 3]

1.4 一些参数函数

import torch.nn as nn

1.4.1 f(x)=Ax+b

?nn.Linear

m = nn.Linear(20,30)

input = autograd.Variable(torch.randn(128,64,20))

output=m(input)

print output.size()

torch.Size([128, 64, 30])

1.4.2 tanh或者Relu

import torch.nn.functional as F

data = autograd.Variable(torch.randn(2,3))

data

Variable containing:
-0.0968 -0.5334  0.5023
 1.6757  1.5588  1.4051
[torch.FloatTensor of size 2x3]

F.relu(data)

Variable containing:
 0.0000  0.0000  0.5023
 1.6757  1.5588  1.4051
[torch.FloatTensor of size 2x3]

F.tanh(data)

Variable containing:
-0.0965 -0.4880  0.4639
 0.9323  0.9152  0.8864
[torch.FloatTensor of size 2x3]

1.4.3 softmax

data = autograd.Variable(torch.randn(2,3,4))

data

Variable containing:
(0 ,.,.) = 
  1.0451  0.2925 -1.0171  1.0889
  0.2728 -0.7194 -0.2669 -0.1076
 -1.3558 -0.1216  0.5279 -2.5866

(1 ,.,.) = 
 -0.5489  1.5895  2.1688 -0.2448
 -0.0070  1.4209 -0.6757  0.0670
  0.4146 -0.3724  1.9020  0.5877
[torch.FloatTensor of size 2x3x4]

F.softmax(data)

Variable containing:
(0 ,.,.) = 
  0.8312  0.2147  0.0397  0.7914
  0.5695  0.1052  0.6008  0.4565
  0.1455  0.5624  0.2020  0.0401

(1 ,.,.) = 
  0.1688  0.7853  0.9603  0.2086
  0.4305  0.8948  0.3992  0.5435
  0.8545  0.4376  0.7980  0.9599
[torch.FloatTensor of size 2x3x4]

F.softmax(data).sum()

Variable containing:
 12
[torch.FloatTensor of size 1]

F.softmax(data).data

(0 ,.,.) = 
  0.8312  0.2147  0.0397  0.7914
  0.5695  0.1052  0.6008  0.4565
  0.1455  0.5624  0.2020  0.0401

(1 ,.,.) = 
  0.1688  0.7853  0.9603  0.2086
  0.4305  0.8948  0.3992  0.5435
  0.8545  0.4376  0.7980  0.9599
[torch.FloatTensor of size 2x3x4]

F.softmax(data).data[0]+F.softmax(data).data[1]

 1  1  1  1
 1  1  1  1
 1  1  1  1
[torch.FloatTensor of size 3x4]

说明是对第0维做了softmax，pytorch第零维在左边