速成tensorflow与pytorch

一、动态图优先：pytorch
动态意味着可以随时查看更改，代码如下：

import torch
W_h=torch.randn(20,20,requires_grad=True)
W_x=torch.randn(20,10,requires_grad=True)
x=torch.randn(1,10)
prev_h=torch.randn(1,20)
h2h=torch.mm(W_h,prev_h.t())
i2h=torch.mm(W_x,x.t())
next_h=h2h+i2h
next_h=next_h.tanh()

常用包：自然语言处理AllenNLP、视觉TorchVision、图计算PyTorch、Fast.ai、部署协议ONNX。
1、GPU加速：
pytorch支持在cuda上用GPU，时间短于CPU。

import torch
import time
print(torch.__version__)
print(torch.cuda.is_available())#查看版本和cuda

x=torch.randn(10000,1000)
y=torch.randn(1000,2000)

t0=time.time()
z=torch.matmul(x,y)
t1=time.time()
print(x.device,t1 - t0,z.norm(2))#CPU时间

device = torch.device("cuda")
x=x.to(device)
y=x.to(device)

t0=time.time()
z=torch.matmul(x,y)
t2=time.time()
print(x.device,t2 - t0,z.norm(2))#第一次cuda需要初始化
t0=time.time()
z=torch.matmul(x,y)
t2=time.time()
print(x.device,t2 - t0,z.norm(2))#GPU时间短于CPU

2、自动求导

import torch
from torch import autograd
x =torch.tensor(1)
W1 =torch.tensor(1.,requires_grad=True)#赋值并求导
W2 =torch.tensor(2.,requires_grad=True)
b =torch.tensor(3.,requires_grad=True)
y=W1**2*x+W2*x+b
print("before:",W1.grad,W2.grad,b.grad)#求梯度，为None(只有浮点dtype的张量需要梯度)
grads=autograd.grad(y,[W1,W2,b])
print("after:",grads[0],grads[1],grads[2])#给定偏微分求梯度，可求出导数

3、神经网络搭建
梯度下降算法：
①求导数对应的为△x，每次x’=x-△x*learning_rate,多次迭代；learning_rate常设为0.001；简单时0.01
②求解器——方向是否保持一致，常见的有sgd、rmspro、pAdam
③闭范式求解——精确；采样——噪声误差（高斯误差被大数据稀释掉）
④用最小二乘法求解yi与wxi+b的最小误差平方和的i总和,为成本函数——《凸优化》
⑤w、b、loss构建三维坐标，但w维度高

import numpy as np
def loss(w,b,points):#定义损失函数
    loss=0
    N = float(len(points))
    for i in range(0,len(points)):
        x = points[i, 0]
        y = points[i, 1]
        loss+=(w*x+b-y)**2
        return loss/N

def step_gradient(current_w,current_b,points,learning_rate):#每一步梯度下降
    gradient_w=0
    gradient_b=0
    N=float(len(points))
    for i in range(0,len(points)):
        x=points[i,0]
        y=points[i,1]
        gradient_w +=(2/N)*x*((current_w*x+current_b)-y)
        gradient_b +=(2/N)*((current_w*x+current_b)-y)
    new_w=current_w-gradient_w*learning_rate
    new_b=current_b-gradient_b*learning_rate
    return [new_w,new_b]

def iter_gradient(start_w,start_b,points,learning_rate,iter_num):#迭代算梯度下降
    last_w=start_w
    last_b=start_b
    for i in range(iter_num):
        last_w,last_b=step_gradient(last_w,last_b,np.array(points),learning_rate)
    return [last_w,last_b]

def run():#运行迭代
    points=np.genfromtxt("file.csv",delimiter=",")
    learning_rate=0.0001
    start_w=0
    start_b=0
    iter_num=1000
    print("starting w={1},b={0},error={2}".format(start_w,start_b,loss(start_w,start_b,points)))
    [last_w,last_b]=iter_gradient(start_w,start_b,points,learning_rate,iter_num)
    print("After {0} inter w={2},b={1},error={3}".format(iter_num,w,b,loss(w,b,points)))

if __name__=='__main__':
    run()

二、静态图优先：tensorflow
静态意味着预先定义，且运行中不能更改，代码如下：

import tensorflow as tf
x_ph=tf.placeholder(tf.int64,name="x")
y_ph=tf.placeholder(tf.int64,name="x")
z_ph=tf.multiply(a_ph,b_ph,name="x*y")
with tf.Session() as sess:
    z_val=sess.run(z_ph,feed_dict={x_ph:[8],y_ph:[9]})
    print(z_val)