TensorFlow笔记(一)

TensorFlow是由谷歌开发的一款分布式深度学习构架,并在2015年11月在GitHub上开元。目前已经发行到了1.0版本,关于如何安装TensorFlow可以参考官方的document或者之前的博文:

http://blog.csdn.net/g8015108/article/details/54882567


核心概念:

TensorFlow作为一种符号化编程,在计算过程中需要首先构建一个有向的计算图,在这个计算图中,每一个运算操作构成一个节点,通过函数关系相连接,而计算的数据则在其中进行流动(flow),这种流动的数据就叫做张量(tensor),这也就是TensorFlow的由来。
对比一下同属于符号化编程的theano和过程化的torch,各有各的优点,torch主要受限于lua,不过随着pytorch的发布,相信torch绝对是TensorFlow的劲敌。

简单的softmax regression

softmax进行手写回归,是机器学习中最基本的一类问题,这里主要用来展示TensorFlow的编写过程。

# -*- coding: utf-8 -*-
"""
Created on Tue Feb 28 22:42:09 2017
@author: sky
"""
import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data/', one_hot=True)
sess = tf.InteractiveSession()
x = tf.placeholder(tf.float32, [None,784])
W = tf.Variable(tf.zeros([784,10]))
b = tf.Variable(tf.zeros([10]))
y = tf.nn.softmax(tf.matmul(x, W) + b)
y_ = tf.placeholder(tf.float32,[None,10])
loss =tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))
train = tf.train.GradientDescentOptimizer(0.5).minimize(loss)
tf.global_variables_initializer().run()
for i in range(1000):
    batch_x, batch_y = mnist.train.next_batch(100)
    train.run({x:batch_x, y_:batch_y})    
correct_pred = tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
print (accuracy.eval({x:mnist.test.images, y_:mnist.test.labels}))

首先从TensorFlow中,导入mnist,也就是
import tensorflow.example.tutorials.mnist
其中one_hot是将标记转化为一个二元向量的flag
接着,我们需要去定义一个会话,这就相当于构建一个TensorFlow的运行环境,建立会话由两种方式:

sess = tf.InteractiveSession 和 sess = tf.Session

两者最大的却别在于InteractiveSession可以在没有指定会话对象的情况下运行变量。
接下来需要构建一个计算图,首先需要定义一个张量x和y_,这个张量是我们以后将要输入的,所以需要用placeholder进行占位,之后需要定义我们的优化变量,用Variable来声明。无论是张量还是优化变量,在声明的时候都需要定义维度和数据类型(float32,int等)。
之后,有了变量,我们就需要像搭积木一样,将这些变量连接在一起(和theano相比真的就是搭积木)。
所以我们需要定义一个连接函数

y = tf.nn.softmax(tf.matmul(x,W)+b)

这样我们就把所有的变量构成了一个图

Created with Raphaël 2.1.0 x y = softmax(wx+b) optimal? output yes no

最后需要用mnist.train.next_batch来读取样本,相当于无放回的抽签样本。并将样本放入对应的之前创建的x和y_中。

从这个例子可以看出,TensorFlow实现算法主要分四步:

1。定义算法公式,也就是前向运行
2。定义loss,并选择优化器(如adam等)
3。迭代训练
4。测试

自编码器

import tensorflow as tf
import sklearn.preprocessing as prep
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data

def xavier_init(n_in, n_out, constant=1):
    low = -constant * np.sqrt(6.0 / (n_in + n_out))
    high = constant * np.sqrt(6.0 / (n_in + n_out))

    return tf.random_uniform((n_in,n_out),
                             minval = low, maxval = high,
                             dtype=tf.float32)

class addGaussianNoiseAutoencoder(object):
    def __init__(self, n_in, n_hidden, activate=tf.nn.relu,
                 optimizer = tf.train.AdamOptimizer(),scale=0.1):
        self.n_in = n_in
        self.n_hidden = n_hidden
        self.activate = activate
        self.scale = tf.placeholder(tf.float32)
        self.training_scale = scale
        def _init_weights(self):
            all_weights = dict()
            all_weights['w1'] = tf.Variable(xavier_init(self.n_in, self.n_hidden))
            all_weights['b1'] = tf.Variable(tf.zeros([self.n_hidden],dtype=tf.float32))
            all_weights['w2'] = tf.Variable(tf.zeros([self.n_hidden,self.n_in],dtype=tf.float32))
            all_weights['b2'] = tf.Variable(tf.zeros([self.n_in],dtype=tf.float32))

        network_weights = self._init_weights()
        self.weights = network_weights

        self.x = tf.placeholder(tf.float32,[None,self.n_in])
        self.hidden = self.activate(tf.add(tf.matmul(self.x - scale * tf.random_normal((n_in,)),
                                                     self.weights['w1']),self.weight['b1']))

        self.re = tf.add(tf.matmul(self.hidden,self.weights['w2']),self.weights['b2'])

        self.cost = 0.5*tf.reduce_sum(tf.pow(tf.subtract(self.re,self.x),2.0))

        self.optimizer = optimizer.minimize(self.cost)
        init = tf.global_variables_initializer()
        self.sess = tf.Session()
        self.sess.run(init)

MLP

# -*- coding: utf-8 -*-
"""
Created on Sat Mar 11 19:29:26 2017

@author: Sky_Gao
"""

from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
mnist = input_data.read_data_sets("MNIST_data/",one_hot=True)
sess = tf.InteractiveSession()

n_in = 784
n_hidden = 500
n_out = 10
w1 = tf.Variable(tf.truncated_normal([n_in,n_hidden],stddev=0.1))
b1 = tf.Variable(tf.zeros([n_hidden]))
w2 = tf.Variable(tf.zeros([n_hidden,n_out]))
b2 = tf.Variable(tf.zeros([n_out]))

x = tf.placeholder(tf.float32,[None,784])
keep_prob = tf.placeholder(tf.float32)

hidden1 = tf.nn.relu(tf.matmul(x,w1) + b1)
hidden1_drop = tf.nn.dropout(hidden1, keep_prob)

y = tf.nn.softmax(tf.matmul(hidden1_drop, w2) + b2)
y_ = tf.placeholder(tf.float32,[None,10])

loss = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y),reduction_indices=[1]))
train = tf.train.AdagradOptimizer(0.3).minimize(loss)

tf.global_variables_initializer().run()

for i in range(1000):
    batch_x, batch_y = mnist.train.next_batch(100)
    train.run({x:batch_x,y_:batch_y,keep_prob:0.8})

correct_pred = tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred,tf.float32))

print (accuracy.eval({x:mnist.test.image,y_:mnist.test.labels,keep_prob:1.0}))

这里的代码主要用到了nn.dropout,这里面的keep_prob是保留率,之前一直把他当做的dropoutrate。

之前有用keras,所以想重复一下keras中的mlp,结果在执行RMSprop的时候两层的网络一直无法运行,换做Adam的方法就可以,准确率好keras给出的demo差不多。其他的优化方法,可以参考官方document,训练完后的准确率在98%左右

CNN

# -*- coding: utf-8 -*-
"""
Created on Sat Mar 11 19:52:06 2017

@author: Sky_Gao
"""

from tensorflow.examples.tutorials.mnist import input_data
import tensorflow as tf
mnist = input_data.read_data_sets("MNIST_data/",one_hot=True)
sess = tf.InteractiveSession()

def weight(shape):
    init = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(init)

def bias(shape):
    init = tf.constant(0.1, shape=shape)
    return tf.Variable(init)

def conv2d(x,w):
    return tf.nn.conv2d(x,w,strides=[1,1,1,1], padding='SAME')

def max_pool(x):
    return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1],padding='SAME')

x = tf.placeholder(tf.float32, [None,784])
y_ = tf.placeholder(tf.float32, [None,10])

x_image = tf.reshape(x, [-1,28,28,1])

w_conv1 = weight([5,5,1,32])
b_conv1 = bias([32])
h_conv1 = tf.nn.relu(conv2d(x_image,w_conv1) + b_conv1)
h_pool1 = max_pool(h_conv1)

w_conv2 = weight([5,5,32,64])
b_conv2 = bias([64])
h_conv2 = tf.nn.relu(conv2d(h_pool1,w_conv2) + b_conv2)
h_pool2 = max_pool(h_conv2)

w_1 = weight([7*7*64,1024])
b_1 = bias([1024])
h_flat = tf.reshape(h_pool2,[-1,7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(h_flat, w_1) + b_1)

keep_prob = tf.placeholder(tf.float32)
h_drop = tf.nn.dropout(h_fc1,keep_prob)

w_2 = weight([1024,10])
b_2 = bias([10])
y = tf.nn.softmax(tf.matmul(h_drop,w_2) + b_2)
loss = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y), reduction_indices=[1]))

train = tf.train.AdamOptimizer(0.0001).minimize(loss)

correct_pred = tf.equal(tf.argmax(y,1),tf.argmax(y_,1))
accuracy = tf.reduce_mean(tf.cast(correct_pred,tf.float32))

tf.global_variables_initializer().run()

for i in range(200):
    batch = mnist.train.next_batch(50)
    train_accuracy = accuracy.eval({x:batch[0],y_:batch[1],keep_prob:1.0})
    train.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5})

CNN的代码其实很简单,只要稍微注意一下都不会出错,但是,内存很关键。我在cpu上运行,8g的内存,在不开任何程序的情况下刚刚够用,听了个歌直接死机。

小结

之前一直在用theano,和TensorFlow相比,theano简直就是魔鬼。连softmax都需要自己写,而TensorFlow就像是个低集成化的模块,什么都可以做,而且自由度很高。比theano简直不知道方便到什么程度

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值