CNN

之前看过一些中文方面的介绍，都说TensorFlow是有Tensor，variable ，operation，session这些组成的，可能是中英文翻译的问题，看主流的中文介绍都是讲的云里雾里的，这次直接梳理https://www.tensorflow.org/api_docs/python/的英文介绍，这样对整个认识会更深一步。

一 tf.Tensor(class tf.Tensor)https://www.tensorflow.org/api_docs/python/tf/Tensor（Tensor类）

Represents one of the outputs of an Operation.

A Tensor is a symbolic handle to one of the outputs of an Operation(大部分的operation返回值都是Tensor，比如tf.matmul()). It does not hold the values of that operation's output, but instead provides a means of computing those values in a TensorFlow tf.Session.

This class has two primary purposes:

1. A Tensor can be passed as an input to another Operation. This builds a dataflow connection between operations, which enables TensorFlow to execute an entire Graph that represents a large, multi-step computation.
2. After the graph has been launched in a session, the value of the Tensor can be computed by passing it to tf.Session.run. t.eval() is a shortcut for calling tf.get_default_session().run(t).

该类的几个属性

device: The name of the device on which this tensor will be produced, or None.

dtype: The DType of elements in this tensor.

graph: The Graph that contains this tensor.

name: The string name of this tensor.

Op: The Operation that produces this tensor as an output.

shape: Returns the TensorShape that represents the shape of this tensor.

该类的方法

(1)__init__(op, value_index, dtype) 创建一个新的tensor

Args:

op: An Operation. Operation that computes this tensor.

value_index: An int. Index of the operation's endpoint that produces this tensor.

dtype: A DType. Type of elements stored in this tensor.

(2) consumers()

Args:

Returns a list of Operations that consume this tensor.

(3) eval(feed_dict=None, session=None) 这个是一个feed

Evaluates this tensor in a Session.

Calling this method will execute all preceding operations that produce the inputs needed for the operation that produces this tensor.(给产生这个tensor的op填数据)

N.B. Before invoking Tensor.eval(), its graph must have been launched in a session, and either a default session must be available, or session must be specified explicitly.之前要调用session，run起来

a = tf.Variable(tf.zeros([2]),tf.int32) b = tf.Variable(tf.zeros([2]),tf.int32) c = tf.add(a,b) sess = tf.InteractiveSession() tf.global_variables_initializer().run() print(c.eval(feed_dict = {a:[2,3],b:[4,5]})) sess.run(c,feed_dict = {a:[2,3],b:[4,5]}) #两种写法都可以

Args:

feed_dict: A dictionary that maps Tensor objects to feed values. See tf.Session.run for a description of the valid feed values.

session: (Optional.) The Session to be used to evaluate this tensor. If none, the default session will be used.

Returns:

A numpy array corresponding to the value of this tensor.

(4) get_shape() Alias of Tensor.shape.

(5) set_shape(shape) Updates the shape of this tensor.

This method can be called multiple times, and will merge the given shape with the current shape of this tensor. It can be used to provide additional information about the shape of this tensor that cannot be inferred from the graph alone. For example, this can be used to provide additional information about the shapes of images:

关于tensor的大小

d = tf.constant([[1.0, 0.0], [0.0, 1.0], [1.0, 0.0], [0.0, 1.0]]) print(d.shape)

输出4,2

二 Constant Value Tensorshttps://www.tensorflow.org/api_guides/python/constant_op(本质也是产生tensor)

TensorFlow provides several operations that you can use to generate constants.（注意，这些本质也是Op，产生的是常数或者随机tensor）

1. tf.zeros

2. tf.zeros_like

3. tf.ones

4. tf.ones_like

5. tf.fill

6. tf.constant

7. tf.linspace

8. tf.range

Random Tensors

TensorFlow has several ops that create random tensors with different distributions. The random ops are stateful, and create new random values each time they are evaluated.

The seed keyword argument in these functions acts in conjunction with the graph-level random seed. Changing either the graph-level seed using tf.set_random_seed or the op-level seed will change the underlying seed of these operations. Setting neither graph-level nor op-level seed, results in a random seed for all operations. See tf.set_random_seed for details on the interaction between operation-level and graph-level random seeds.

1.tf.random_normal

2.tf.truncated_normal

3.tf.random_uniform

4.tf.random_shuffle

5.tf.random_crop

6.tf.multinomial

7.tf.random_gamma

8.tf.set_random_seed

三 tf.Variable(class tf.Variable)https://www.tensorflow.org/api_docs/python/tf/Variable (Variable类)

A variable maintains state in the graph across calls to run(). You add a variable to the graph by constructing an instance of the class Variable.（作用和Tensor类不同，Tensor用于Op之间传递参量，变量用于存储值）

1 变量的注意在于初始化

The Variable() constructor requires an initial value for the variable, which can be a Tensor of any type and shape. The initial value defines the type and shape of the variable. After construction, the type and shape of the variable are fixed. The value can be changed using one of the assign methods.

If you want to change the shape of a variable later you have to use an assign Op with validate_shape=False.

Just like any Tensor, variables created with Variable() can be used as inputs for other Ops in the graph.（variables可以作为Op输入） Additionally, all the operators overloaded for the Tensor class are carried over(转移到) to variables, so you can also add nodes to the graph by just doing arithmetic on variables. 这个意思应该Tensor中可以含有变量，变量可以在后期赋值（下例中的tensor和variable是在一起运算的）

When you launch the graph, variables have to be explicitly initialized before you can run Ops that use their value. You can initialize a variable by running its initializer op, restoring the variable from a save file, or simply running an assign Op that assigns a value to the variable. In fact, the variable initializer op is just an assign Op that assigns the variable's initial value to the variable itself.

The most common initialization pattern is to use the convenience function global_variables_initializer() to add an Op to the graph that initializes all the variables. You then run that Op after launching the graph.（这里是add an 初始化op）

If you need to create a variable with an initial value dependent on another variable, use the other variable's initialized_value(). This ensures that variables are initialized in the right order.

2 变量值收集

All variables are automatically collected in the graph where they are created. By default, the constructor adds the new variable to the graph collection GraphKeys.GLOBAL_VARIABLES. The convenience function global_variables() returns the contents of that collection.

When building a machine learning model it is often convenient to distinguish between variables holding the trainable model parameters and other variables such as a global step variable used to count training steps. To make this easier, the variable constructor supports a trainable=<bool> parameter. If True, the new variable is also added to the graph collection GraphKeys.TRAINABLE_VARIABLES. The convenience function trainable_variables() returns the contents of this collection. The various Optimizer classes use this collection as the default list of variables to optimize.

1. 核心属性

(1)initial_value

Returns the Tensor used as the initial value for the variable.

Note that this is different from initialized_value() which runs the op that initializes the variable before returning its value. This method returns the tensor that is used by the op that initializes the variable.

(2) initializer

The initializer operation for this variable.

4 核心方法

(1) __init__(initial_value=None, trainable=True, collections=None, validate_shape=True, caching_device=None, name=None, variable_def=None, dtype=None, expected_shape=None, import_scope=None)

Creates a new variable with value initial_value.

The new variable is added to the graph collections listed in collections, which defaults to [GraphKeys.GLOBAL_VARIABLES].

If trainable is True the variable is also added to the graph collection GraphKeys.TRAINABLE_VARIABLES.

This constructor creates both a variable Op and an assign Op to set the variable to its initial value.

(2) assign(value, use_locking=False)

Assigns a new value to the variable.

This is essentially a shortcut for assign(self, value).

Args:

value: A Tensor. The new value for this variable.

use_locking: If True, use locking during the assignment.

Returns:

A Tensor that will hold the new value of this variable after the assignment has completed.

(3) assign_add(delta, use_locking=False)

Adds a value to this variable.

This is essentially a shortcut for assign_add(self, delta).

Args:

delta: A Tensor. The value to add to this variable.

use_locking: If True, use locking during the operation.

Returns:

A Tensor that will hold the new value of this variable after the addition has completed.

(4)assign_sub(delta, use_locking=False)

Subtracts a value from this variable.

This is essentially a shortcut for assign_sub(self, delta).

Args:

delta: A Tensor. The value to subtract from this variable.

use_locking: If True, use locking during the operation.

Returns:

A Tensor that will hold the new value of this variable after the subtraction has completed.

(5) eval(session=None)

In a session, computes and returns the value of this variable. 计算或者返回variable的值

This is not a graph construction method, it does not add ops to the graph.(这一点要注意)

This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See tf.Session for more information on launching a graph and on sessions.

Args:

session: The session to use to evaluate this variable. If none, the default session is used.

Returns:

A numpy ndarray with a copy of the value of this variable.

(6) initialized_value()

Returns the value of the initialized variable.

You should use this instead of the variable itself to initialize another variable with a value that depends on the value of this variable.

Beware of using initialized_value except during initialization: initialized_value causes the Variable's initializer op to be run, so running this op resets the variable to the initial value.

Returns:

A Tensor holding the value of this variable after its initializer has run.

(7)load(value, session=None)

Load new value into this variable

Writes new value to variable's memory. Doesn't add ops to the graph.

This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used. See tf.Session for more information on launching a graph and on sessions.

Args:

value: New variable value

session: The session to use to evaluate this variable. If none, the default session is used.

(7) read_value()

Returns the value of this variable, read in the current context.

Can be different from value() if it's on another device, with control dependencies, etc.

Returns:

A Tensor containing the value of the variable.

四 tf.placeholder(dtype, shape=None, name=None)https://www.tensorflow.org/api_docs/python/tf/placeholder

这个其实是个feed

Inserts a placeholder for a tensor that will be always fed.

Important: This tensor will produce an error if evaluated. Its value must be fedusing the feed_dict optional argument to Session.run(), Tensor.eval(), or Operation.run().

Args:

dtype: The type of elements in the tensor to be fed.

shape: The shape of the tensor to be fed (optional). If the shape is not specified,you can feed a tensor of any shape.

name: A name for the operation (optional).

Returns:

A Tensor that may be used as a handle for feeding a value, but not evaluated directly.

五 一个简单的CNN

其中可以发现对于数据集而言，用的是placeholder,对于权值、偏置用的是variable参量。中间op返回的数据都是Tensor

# coding: utf-8 # In[1]: from tensorflow.examples.tutorials.mnist import input_data import tensorflow as tf # In[2]: mnist = input_data.read_data_sets("MNIST_data/",one_hot = True) sess = tf.InteractiveSession() # In[3]: #########定义初始化函数############### def weight_variable(shape): #先定义好初始化函数以便重复使用。我们需要给权重制造一些随机噪声来打破完全对称 initial = tf.truncated_normal(shape,stddev = 0.1) #返回A tensor of the specified shape filled with random truncated normal values. return tf.Variable(initial) #返回一个Tensor.Returns the Tensor used as the initial value for the variable. def bias_variable(shape): initial = tf.constant(0.1,shape = shape) #返回A Constant Tensor. return tf.Variable(initial) # In[4]: #########定义神经网络中共的核心函数############### def conv2d(x,W): #W卷积参数[5,5,1,32] 5*5代表卷积核尺寸；1：代表通道数,灰度图像为1，RGB图像为3；32卷积核数目 return tf.nn.conv2d(x,W,strides = [1,1,1,1],padding = "SAME") #返回A Tensor. Has the same type as input def max_pool_2_2(x): #这里使用2*2的最大池化，即将一个2*2变成1*1.因为希望整体缩小图片尺寸，所以strides设置为横竖两个方向以2为步长。 return tf.nn.max_pool(x,ksize=[1,2,2,1],strides = [1,2,2,1],padding = "SAME") #返回A Tensor with type tf.float32. The max pooled output tensor. # In[5]: #########在正式设计神经网络之前先定义输入的placeholder函数########### x = tf.placeholder(tf.float32,[None,784]) #输入样本参量 y_ = tf.placeholder(tf.float32,[None,10]) #输入样本标签,输出A Tensor that may be used as a handle for feeding a value, but not evaluated directly. x_image = tf.reshape(x,[-1,28,28,1])#-1代表样本数量不确定，每个样本28*28，1代表颜色通道数 # In[6]: ###############定义第一个卷积层##################################### W_conv1 = weight_variable([5,5,1,32]) #卷积核尺寸5*5,1个颜色通道，32个不同卷积核,感觉像高维向量 b_conv1 = bias_variable([32]) h_conv1 = tf.nn.relu(conv2d(x_image,W_conv1) + b_conv1) h_pool1 = max_pool_2_2(h_conv1) ###############定义第二个卷积层##################################### W_conv2 = weight_variable([5,5,32,64]) #卷积核尺寸5*5,注意这里变为32个通道，因为第一层输出32个卷积核结果，64个不同卷积核,感觉像高维向量 b_conv2 = bias_variable([64]) h_conv2 = tf.nn.relu(conv2d(h_pool1,W_conv2) + b_conv2) h_pool2 = max_pool_2_2(h_conv2) ###############最后跟一个全连接层#################### #前面经过两次池化，所以边长只有原来1/4了，图片尺寸为7*7，第二层卷积核数量为64，其输出tensor为7*7*64,隐含节点数为1024。 W_fc1 = weight_variable([7*7*64,1024]) #不知道这里7*7*64怎么来的 b_fc1 = bias_variable([1024]) h_pool2_flat = tf.reshape(h_pool2,[-1,7*7*64]) h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1) ##################加一个Dropout层#################### keep_prob = tf.placeholder(tf.float32) h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob) ##################最后将Dropout层连接一个Softmax层######## W_fc2 = weight_variable([1024,10]) b_fc2 = bias_variable([10]) y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop,W_fc2) + b_fc2) # In[7]: ###########定义损失函数,######################## cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv),reduction_indices=[1])) train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) ##########评测准确率################################ correct_prediction = tf.equal(tf.argmax(y_conv,1),tf.argmax(y_,1)) accuracy = tf.reduce_mean(tf.cast(correct_prediction,tf.float32)) # In[8]: #########################训练################ tf.global_variables_initializer().run() for i in range(20000): batch = mnist.train.next_batch(50) if i%100 == 0: train_accuracy = accuracy.eval(feed_dict = {x:batch[0],y_:batch[1],keep_prob:1.0}) print("step %d, training accuracy %g" %(i,train_accuracy)) train_step.run(feed_dict={x:batch[0],y_:batch[1],keep_prob:0.5}) print("test accuracy %g" %accuracy.eval(feed_dict = {x:mnist.test.images,y_:mnist.test.labels,keep_prob:1.0})) #eval参见 tf.Tensor