深度学习笔记——深度学习框架TensorFlow（一）

一. 学习网站：

1. Introduction：https://www.tensorflow.org/versions/r0.12/get_started/index.html
2. Tutorials：https://www.tensorflow.org/versions/r0.12/tutorials/index.html
3. API：https://www.tensorflow.org/versions/r0.12/api_docs/python/index.html

二. Introduction：

1. 网站：https://www.tensorflow.org/versions/r0.12/get_started/index.html
2. 正文（中英对照版）：

Let’s get you up and running with TensorFlow!
让我们开始学习TensorFlow！

But before we even get started, let’s peek at what TensorFlow code looks like in the Python API, so you have a sense of where we’re headed.
在我们学习之前，让我们看一下TensorFlow的代码在Python API中是怎么样的，以便你知道接下来我们会如何学习

Here’s a little Python program that makes up some data in two dimensions, and then fits a line to it.
首先，我们看一个Python程序，这个Python程序描述的是在二维空间中的多个点，然后寻找一条最合适的线以适配这些点。

import tensorflow as tf
import numpy as np
#Create 100 phony x,y data points in Numpy,y = x*0.1+0.3
x_data = np.random.rand(100).astype(np.float32)
y_data = x_data*0.1+0.3
#Try to find values for W and b that compute y_data = W*x_data+b
#y_data = W*x_data+b
#we know W should be 0.1 and b 0.3
W = tf.Variable(tf.random_uniform([1],-1.0,1.0))
b = tf.Variable(tf.zeros([1]))
y = W*x_data+b
#Minimize the mean squared errors
loss = tf.reduce_mean(tf.square(y-y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)
#Before starting,initialize the variables,We will run this first
init = tf.global_variables_initializer()
#Launch the graph
sess = tf.Session()
sess.run(init)
#Fit the line
for step in range(201):
    sess.run(train)
    if step % 20 == 0:
      print(step,sess.run(W),sess.run(b))
#Learns best fit is W:[0.1],b:[0.3]

The first part of this code builds the data flow graph. TensorFlow does not actually run any computation until the session is created and the run function is called.
第一部分代码构建了数据流图，TensorFlow直到session被创建以及run函数被调用才真正开始执行。

To whet your appetite further, we suggest you check out what a classical machine learning problem looks like in TensorFlow. In the land of neural networks the most “classic” classical problem is the MNIST handwritten digit classification. We offer two introductions here, one for machine learning newbies, and one for pros. If you’ve already trained dozens of MNIST models in other software packages, please take the red pill. If you’ve never even heard of MNIST, definitely take the blue pill. If you’re somewhere in between, we suggest skimming blue, then red.
为了更进一步吸引你的兴趣，我们建议你看一下机器学习的基本问题在TensorFlow中的应用。在机器学习领域最经典的焦点问题是MNIST手写数字识别。在这我们提供2个介绍：第一个给新手看，第二给专业人士看。如果你已经训练了多个MNIST模型，那么请点击红色药丸。如果你你之前没听过MNIST，则你需要点击下面的蓝色药丸。如果你介于两者之间，我们建议你点击蓝色药丸，再点击红色药丸。
红色药丸链接 : https://www.tensorflow.org/versions/r0.12/tutorials/mnist/pros/index.html
蓝色药丸链接：https://www.tensorflow.org/versions/r0.12/tutorials/mnist/beginners/index.html

If you’re already sure you want to learn and install TensorFlow you can skip these and charge ahead. Don’t worry, you’ll still get to see MNIST – we’ll also use MNIST as an example in our technical tutorial where we elaborate on TensorFlow features.
如果你确定你要学习TensorFlow，你可以跳过上面这些，直接往前看。别担心，你还是会接触到MNIST——我们也会以MNIST为例阐述TensorFlow的一些特征。

建议的步骤：
Download and Setup：https://www.tensorflow.org/versions/r0.12/get_started/os_setup.html
Basic Usage：https://www.tensorflow.org/versions/r0.12/get_started/basic_usage.html
TensorFlow Mechaincs 101：https://www.tensorflow.org/versions/r0.12/tutorials/mnist/tf/index.html
Tinker with a neural network in your browser：http://playground.tensorflow.org/#activation=tanh&batchSize=10&dataset=circle&regDataset=reg-plane&learningRate=0.03&regularizationRate=0&noise=0&networkShape=4,2&seed=0.41777&showTestData=false&discretize=false&percTrainData=50&x=true&y=true&xTimesY=false&xSquared=false&ySquared=false&cosX=false&sinX=false&cosY=false&sinY=false&collectStats=false&problem=classification&initZero=false&hideText=false##

三. Basic Usage：

1. 参考网站：https://www.tensorflow.org/versions/r0.12/get_started/basic_usage.html#overview
2. 正文（中英对照）：
   Basic Usage：
   To use TensorFlow you need to understand how TensorFlow:

Represents computations as graphs.
Executes graphs in the context of Sessions.
Represents data as tensors.
Maintains state with Variables.
Uses feeds and fetches to get data into and out of arbitrary operations.
使用TensorFlow之前，你需要了解TensorFlow：
1. 以图的方式来表示估计
2. 通过Sessions在上下文中执行图形
3. 通过张量表示数据
4. 通过Variables保持状态
5. 使用提要和抓取来获得数据，并且输出任意操作。
OverView：
TensorFlow is a programming system in which you represent computations as graphs. Nodes in the graph are called ops (short for operations). An op takes zero or more Tensors, performs some computation, and produces zero or more Tensors. In TensorFlow terminology, a Tensor is a typed multi-dimensional array. For example, you can represent a mini-batch of images as a 4-D array of floating point numbers with dimensions [batch, height, width, channels].
TensorFlow是一个程序系统，在这系统中你可以将计算用图进行表示，途中的结点被称为操作，一个操作可能会占据0或者更多的张量，可以用来执行一些计算。张量是一个典型的多维数组。例如，你可以用4维float类型数组的值来表示一个小批次图形[batch,height,width,channels]

A TensorFlow graph is a description of computations. To compute anything, a graph must be launched in a Session. A Session places the graph ops onto Devices, such as CPUs or GPUs, and provides methods to execute them. These methods return tensors produced by ops as numpy ndarray objects in Python, and as tensorflow::Tensor instances in C and C++.
一个tensorflow图是一种描述计算。无论计算什么都必须在一个会话中启动一个图表。会话将在图形操作安置在设备上，如CPU或GPU，并且提供一个思想以执行它们。这些方法返回由在Python中操作的numpy ndarray 对象的张量，并为tensorflow：：在C和C++的张量的例子。

The computation graph：
TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.
tensorflow程序通常分为施工阶段，组装图，和执行阶段，使用一个会话中执行行动图。

For example, it is common to create a graph to represent and train a neural network in the construction phase, and then repeatedly execute a set of training ops in the graph in the execution phase.
例如，在施工阶段创建一个图形以表示和训练一个神经网络是非常常见的。接下来在施工阶段在图形中分别执行一系列训练操作。

TensorFlow can be used from C, C++, and Python programs. It is presently much easier to use the Python library to assemble graphs, as it provides a large set of helper functions not available in the C and C++ libraries.
TensorFlow可以使用到C/C++/Python程序中。现在它能更简单的使用Python库进行图表收集，因为它通过了一系列帮助函数，而这些在C/C++库中是没有的。

The session libraries have equivalent functionalities for the three languages.
会话库对于三种语言有等效功能。

Building the graph：
To build a graph start with ops that do not need any input (source ops), such as Constant, and pass their output to other ops that do computation.
以操作为始构建一个图形不需要任何输入(源操作)，例如Constant(常熟)，以及将它们输出给其他操作用作计算。

The ops constructors in the Python library return objects that stand for the output of the constructed ops. You can pass these to other ops constructors to use as inputs.
在Python库中这个操作结构将中返回一个对象，这个对象代表这个操作的输出。其他操作结构可以将这输出当作是一个输入。

The TensorFlow Python library has a default graph to which ops constructors add nodes. The default graph is sufficient for many applications. See the Graph class documentation for how to explicitly manage multiple graphs.
TensorFlow的Python库有一个默认图形，这里的操作构造函数可以增加结点。默认图表对于许多应用来说是足够的了。看图表类文档（https://www.tensorflow.org/versions/r0.12/api_docs/python/framework.html#Graph）如何显式管理多个图形。

import tensorflow as tf
#Create a constatnt op that produces a 1*2 matrix,the op is added as a node to the default graph
#the value returned by the constructor represents the output of the Constant op.
matrix1 = tf.constant([[3.,3.]])
#Create another Constant that produces a 2*1 matrix
matrix2 = tf.constant([[2.],[2.]])
#Create a Matmul op that takes 'matrix1' and 'matrix2' as inputs.
#The returned value,'product',represents the result of the matrix multiplication
product = tf.matmul(matrix1,matrix2)

The default graph now has three nodes: two constant() ops and one matmul() op. To actually multiply the matrices, and get the result of the multiplication, you must launch the graph in a session.
现在这个默认图形有三个结点，两个constant()操作，一个matmul()操作。为了得到矩阵相乘的结果，你必须在会话（session）中启动图形

Launching the graph in a session：
Launching follows construction. To launch a graph, create a Session object. Without arguments the session constructor launches the default graph.
为了启动一个图表，我们需要创建一个会话（Session）对象，毫无疑问的是会话构造函数需要启动一个默认的图标。

See the Session class for the complete session API.
详细的请参见session的API（https://www.tensorflow.org/versions/r0.12/api_docs/python/client.html#session-management）

#Launch the default graph
sess = tf.Session()
#To run the matmul op we call the session 'run()'method，passing 'product'
#which represents the output of the matmul op.  This indicates to the call
#that we want to get the output of the matmul op back.

#All inputs needed by the op are run automatically by the session.  They
#typically are run in parallel.

#The call 'run(product)' thus causes the execution of three ops in the
#graph: the two constants and matmul.

result = sess.run(product)
print(result)
#Close the Session when we're done
sess.close()

Sessions should be closed to release resources. You can also enter a Session with a “with” block. The Session closes automatically at the end of the with block.
会话应该在释放资源时被关闭，你可以可以输入一个会话以一个”with”块，在with块的结尾会话会自动关闭：

with tf.Session() as sess:
    result = sess.run([product])
    print(result)

The TensorFlow implementation translates the graph definition into executable operations distributed across available compute resources, such as the CPU or one of your computer’s GPU cards. In general you do not have to specify CPUs or GPUs explicitly. TensorFlow uses your first GPU, if you have one, for as many operations as possible.
TensorFlow的实现可以转换成可执行的操作的图形定义分布在可用的计算资源，如CPU或计算机的GPU。通常你不必明确指定CPU或GPU。如果你有一个GPU，为了尽可能尽可能多的操作，tensorflow会使用你的第一个。

If you have more than one GPU available on your machine, to use a GPU beyond the first you must assign ops to it explicitly. Use with…Device statements to specify which CPU or GPU to use for operations:
如果你的服务器中有超过一个的可用GPU，你必须显式的标明需要使用的GPU。通过设备陈述去明确使用的是CPU还是GPU。

with tf.Session() as sess:
    with tf.device("/gpu:1")
        matrix1 = tf.constant([[3.,3.]])
        matrix2 = tf.constant([[2.][2.]])
        product = tf.matmul(matrix1,matrix2)        

Devices are specified with strings. The currently supported devices are:
设备被描述成字符串，当前支持的设备描述如下：

"/cpu:0": The CPU of your machine
"/gpu:0"：The GPU of your machine,if you have one.
"/gpu:1"：The second GPU of your machine,etc.

See Using GPUs for more information about GPUs and TensorFlow.
看使用GPU的更多知识请参考GPUs和TensorFlow(https://www.tensorflow.org/versions/r0.12/how_tos/using_gpu/index.html)

Launching the graph in a distributed session:
To create a TensorFlow cluster, launch a TensorFlow server on each of the machines in the cluster. When you instantiate a Session in your client, you pass it the network location of one of the machines in the cluster:
创建一个TensorFlow集群，在每台机器的集群上登录一个TensorFlow服务器。当你实例化一个Session在你的客户端时，你通过它传递集群中的一台机器的网络位置。

with tf.Session("grpc://example.org:2222") as session:
    #Calls to sess.run(...) will be executed on the cluster.

This machine becomes the master for the session. The master distributes the graph across other machines in the cluster (workers), much as the local implementation distributes the graph across available compute resources within a machine.
这个机器会变成会话中的主流，主流通过集群(工人)中的其他机器分配图表，正如本地实现将图表分布在机器的可用的计算资源上一样。

You can use “with tf.device():” statements to directly specify workers for particular parts of the graph:
你可以使用”with tf.device():”陈述直接明确的指定工人(集群)为某个特定部分的图表工作。

with tf.device("/job:ps/task:0")
    weights = tf.Variable(...)
    biases = tf.Variable(...)

See the Distributed TensorFlow How To for more information about distributed sessions and clusters.
获取更多分布式TensorFlow如何为分布式会话和集群工作的更多信息。（https://www.tensorflow.org/versions/r0.12/how_tos/distributed/index.html）

Interactive Usage:
The Python examples in the documentation launch the graph with a Session and use the Session.run() method to execute operations.
文档中Python的案例通过Session生成一个图表，通过Session.run()方法执行一个操作。

For ease of use in interactive Python environments, such as IPython you can instead use the InteractiveSession class, and the Tensor.eval() and Operation.run() methods. This avoids having to keep a variable holding the session.
对于使用交互式Python的环境，比如IPython，你可以转而使用InteractiveSession类，和Tensor.eval()和Operation.run()方法。这样就避免了保留会话的变量。

#Enter an interative TensorFlow Session.
import tensorflow as tf
sess = tf.InteractiveSession()

x = tf.Variable([1.0,2.0])
a = tf.constant([3.0,3.0])

#Initialize 'x' using the run() method of its initializer op.
#Add an op to subtract 'a' from 'x'.Run it and print the result.
sub = tf.subtract(x,a)
print(sub.eval())
#==>[-2.-1.]
#Close the Session when we're done.
sess.close()

Tensors:
TensorFlow programs use a tensor data structure to represent all data – only tensors are passed between operations in the computation graph. You can think of a TensorFlow tensor as an n-dimensional array or list. A tensor has a static type, a rank, and a shape. To learn more about how TensorFlow handles these concepts, see the Rank, Shape, and Type reference.
tensorflow程序使用张量数据结构来表示所有的数据–计算图中操作之间传递的是张量。你可以想象一个tensorflow张量为n维数组或列表。张量有静态类型、秩和形状。要了解更多关于TensorFlow如何处理这些概念，看排名，形状和类型的引用。（https://www.tensorflow.org/versions/r0.12/resources/dims_types.html）

Variables:
变量（Variables）通过图表执行维持状态，下面的示例显示一个用作简单计数器的变量。看更多Variables细节（https://www.tensorflow.org/versions/r0.12/how_tos/variables/index.html）

#Create a Variable,that will be initialized to the scalar value 0.
state = tf.Variable(0,name="counter")
#Create an Op to add one to 'state'
one = tf.constant(1)
new_value = tf.add(state,one)
update = tf.assign(state,new_value)
#Variables must be initialized by running an 'init' Op after having launched the graph.We first have to add the 'init' Op to the graph.
init_op = tf.global_variables_initializer()
#Launch the graph and run the ops.
with tf.Session() as sess:
    #Run the 'init' op
    sess.run(init_op)
    #Print the initial value of 'state'
    print(sess.run(state))
    #Run the op that updates 'state' and print 'state'.
    for _ in range(3):
        sess.run(update)
        print(sess.run(state))
# output:
#0
#1
#2
#3

The assign() operation in this code is a part of the expression graph just like the add() operation, so it does not actually perform the assignment until run() executes the expression.
代码中的assign()操作是图所描述的表达式的一部分，如同add()操作。所以在run()执行之前它并不能真正得到执行。

You typically represent the parameters of a statistical model as a set of Variables. For example, you would store the weights for a neural network as a tensor in a Variable. During training you update this tensor by running a training graph repeatedly.
通常将统计模型的参数表示为一组变量。例如，你可以存储一个神经网络的权重作为变量中的一个张量。在训练过程中，通过反复运行一个训练图来更新这个张量。

Fetches:
To fetch the outputs of operations, execute the graph with a run() call on the Session object and pass in the tensors to retrieve. In the previous example we fetched the single node state, but you can also fetch multiple tensors:
为了取回操作的输出内容，可以在通过Session对象的run()方法执行图像时，传入一些tensor，这些tensor会帮助你取回结果。在先前的例子中，我们不仅可以获取单独的点状态，也可以获取多个tensors。

input1 = tf.constant([3.0])
input2 = tf.constant([2.0])
input3 = tf.constant([5.0])
intermed = tf.add(input2,input3)
mul = tf.multiply(input1,intermed)
with tf.Session() as sess:
    result = sess.run([mul,intermed])
    print(result)
#output:
#[array([21.],dtype=float32),array([7.],dtype=float32)]

All the ops needed to produce the values of the requested tensors are run once (not once per requested tensor).
需要获取的多个 tensor 值，在 op 的一次运行中一起获得（而不是逐个去获取 tensor）。

Feeds:
The examples above introduce tensors into the computation graph by storing them in Constants and Variables. TensorFlow also provides a feed mechanism for patching a tensor directly into any operation in the graph.
上述示例在计算图中引入了 tensor，并以常量或变量的形式存储。TensorFlow也提供feed机制，用来直接将tensor修补成图像中的任意操作。

A feed temporarily replaces the output of an operation with a tensor value. You supply feed data as an argument to a run() call. The feed is only used for the run call to which it is passed. The most common use case involves designating specific operations to be “feed” operations by using tf.placeholder() to create them:
一个feed通过一个tensor值，暂时性取代一个操作的输出，提供一个feed数据作为run()的参数，eed 只在调用它的方法内有效，在方法结束后他就会消失。最常见的用例是将某些特殊的操作指定为 “feed” 操作, 标记的方法是使用 tf.placeholder() 为这些操作创建占位符。

input1 = tf.placeholder(tf.float32)
input2 = tf.placeholder(tf.float32)
output = tf.mul(input1,input2)
with tf.Session() as sess:
    print(sess.run([output],feed_dict={input1:[7.],input2:[2.]}))

A placeholder() operation generates an error if you do not supply a feed for it. See the MNIST fully-connected feed tutorial (source code) for a larger-scale example of feeds.
如果没有提供足够的feed，那么在使用placeholder()操作时将会发生错误，参考以下资料（资料：https://www.tensorflow.org/versions/r0.12/tutorials/mnist/tf/index.html /源码：https://github.com/tensorflow/tensorflow/blob/r0.12/tensorflow/examples/tutorials/mnist/fully_connected_feed.py）