《Tensorflow》Getting Started with Tensorflow

本文是参考并分析Tensorflow官网上的学习教程，谨以日后Review

学习链接：https://www.tensorflow.org/get_started/get_started

tf.train API

这里从训练一个线性回归模型来介绍tf.train API的使用

代码

# Date: 2017-07-01
# Author: Amusi
# Content: 在官网上学习tensorflow基础教程(二): 机器学习训练
# url: https://www.tensorflow.org/get_started/get_started#the_computational_graph/

# Summary: The completed trainable linear regression model
# calculate the w(should be -1) and b(should be 1)

import numpy as np
import tensorflow as tf

print(tf.__version__)   # the version of tensorflow

# Model parameters
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W * x + b
y = tf.placeholder(tf.float32)
# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
# training data
x_train = [1,2,3,4]
y_train = [0,-1,-2,-3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
  sess.run(train, {x:x_train, y:y_train})

# evaluate training accuracy
curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x:x_train, y:y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

分析

概述：上述代码给定一组x和y点坐标集，试图用linear_model(y)= W*x + b来拟合该曲线（坐标点集）

变量W和b是float类型，分别代表了代数中的斜率和截距

x和y是模型输入和输出变量（float类型）

loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares

loss是损失值，即拟合模型与训练数据的拟合误差损失

optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)

用优化器（optimizer）中的 梯度下降算法的优化器tf.train.GradientDescentOptimizer，实现对模型的优化

这里的学习率为0.01

x_train = [1,2,3,4]
y_train = [0,-1,-2,-3]

x_train和y_train是训练数据

init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
  sess.run(train, {x:x_train, y:y_train})

上述代码是循环训练1000次，生成训练模型

curr_W, curr_b, curr_loss = sess.run([W, b, loss], {x:x_train, y:y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

使用sess.run来评估训练的准确率（accuracy）

运行结果

W: [-0.9999969] b: [ 0.99999082] loss: 5.69997e-11

上述W（斜率），训练结果是-0.9999969，接近-1；b（截距）的训练结果是0.99999082，接近1；损失值为5.699973e-11，接近0

可见损失值loss非常小。

tf.contrib.learn

tf.contrib.learn是一个高级Tensorflow函数库，其简化机器学习的机制，包括：

运行训练循环（running training loops）

运行估计循环（running evaluation loops）

管理数据集（managing data sets）

管理供给数据（managing feeding）

基本用法

注意，tf.contrib.learn可以简化线性回归程序

代码

import tensorflow as tf
# NumPy is often used to load, manipulate and preprocess data.
import numpy as np

# Declare list of features. We only have one real-valued feature. There are many
# other types of columns that are more complicated and useful.
features = [tf.contrib.layers.real_valued_column("x", dimension=1)]

# An estimator is the front end to invoke training (fitting) and evaluation
# (inference). There are many predefined types like linear regression,
# logistic regression, linear classification, logistic classification, and
# many neural network classifiers and regressors. The following code
# provides an estimator that does linear regression.
estimator = tf.contrib.learn.LinearRegressor(feature_columns=features)

# TensorFlow provides many helper methods to read and set up data sets.
# Here we use two data sets: one for training and one for evaluation
# We have to tell the function how many batches
# of data (num_epochs) we want and how big each batch should be.
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])
input_fn = tf.contrib.learn.io.numpy_input_fn({"x":x_train}, y_train,
                                              batch_size=4,
                                              num_epochs=1000)
eval_input_fn = tf.contrib.learn.io.numpy_input_fn(
    {"x":x_eval}, y_eval, batch_size=4, num_epochs=1000)

# We can invoke 1000 training steps by invoking the  method and passing the
# training data set.
estimator.fit(input_fn=input_fn, steps=1000)

# Here we evaluate how well our model did.
train_loss = estimator.evaluate(input_fn=input_fn)
eval_loss = estimator.evaluate(input_fn=eval_input_fn)
print("train loss: %r"% train_loss)
print("eval loss: %r"% eval_loss)

代码分析

x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])

x_train和y_train是训练（training）数据

x_eval和y_eval是评估（evaluation）数据

estimator.fit(input_fn=input_fn, steps=1000)

调用（invoke）1000次训练循环

train_loss = estimator.evaluate(input_fn=input_fn)
eval_loss = estimator.evaluate(input_fn=eval_input_fn)

计算训练和评估损失值

运行结果

train loss: {'global_step': 1000, 'loss': 4.3049088e-08}
eval loss: {'global_step': 1000, 'loss': 0.0025487561}

训练和评估损失值都很小，可见训练模型有效

自定义模型

假设我们想要创建一个Tensorflow未集成的自定义模型，我们仍然可以保留（retain）tf.contrib.learn的数据集、供给数据（feeding）、训练等抽象。例如，我们可以用自己所学的知识来实现一个等价于线性回归（LinearRegressor）的模型

为了结合tf.contrib.learn来自定义模型，我们需要使用tf.contrib.learn.Estimator。tf.contrib.learn.LinearRegressor实质上是tf.contrib.learn.Estimator的子类。为了替代子类，我们只提供Estimator的model_fn函数来告诉tf.contrib.learn如何估计预测，训练步骤和损失值。

代码

官方代码（有问题）

import numpy as np
import tensorflow as tf
# Declare list of features, we only have one real-valued feature
def model(features, labels, mode):
  # Build a linear model and predict values
  W = tf.get_variable("W", [1], dtype=tf.float64)
  b = tf.get_variable("b", [1], dtype=tf.float64)
  y = W*features['x'] + b
  # Loss sub-graph
  loss = tf.reduce_sum(tf.square(y - labels))
  # Training sub-graph
  global_step = tf.train.get_global_step()
  optimizer = tf.train.GradientDescentOptimizer(0.01)
  train = tf.group(optimizer.minimize(loss),
                   tf.assign_add(global_step, 1))
  # ModelFnOps connects subgraphs we built to the
  # appropriate functionality.
  return tf.contrib.learn.ModelFnOps(
      mode=mode, predictions=y,
      loss=loss,
      train_op=train)

estimator = tf.contrib.learn.Estimator(model_fn=model)
# define our data sets
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])
input_fn = tf.contrib.learn.io.numpy_input_fn({"x": x_train}, y_train, 4, num_epochs=1000)

# train
estimator.fit(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did. 
train_loss = estimator.evaluate(input_fn=input_fn)
eval_loss = estimator.evaluate(input_fn=eval_input_fn)
print("train loss: %r"% train_loss)
print("eval loss: %r"% eval_loss)

修改后的代码

# Date: 2017-07-18
# Author: Amusi
# Content: 在官网上学习tensorflow基础教程(三): 机器学习训练
# url: https://www.tensorflow.org/get_started/get_started

# Summary: A custom model
# The custom model that shows how to implement our own equivalent model to LinearRegressor using our knowledge of the lower level TensorFlow API.
import numpy as np
import tensorflow as tf
# Declare list of features, we only have one real-valued feature
def model(features, labels, mode):
  # Build a linear model and predict values
  W = tf.get_variable("W", [1], dtype=tf.float64)
  b = tf.get_variable("b", [1], dtype=tf.float64)
  y = W*features['x'] + b
  # Loss sub-graph
  loss = tf.reduce_sum(tf.square(y - labels))
  # Training sub-graph
  global_step = tf.train.get_global_step()
  optimizer = tf.train.GradientDescentOptimizer(0.01)
  train = tf.group(optimizer.minimize(loss),
                   tf.assign_add(global_step, 1))
  # ModelFnOps connects subgraphs we built to the
  # appropriate functionality.
  return tf.contrib.learn.ModelFnOps(
      mode=mode, predictions=y,
      loss=loss,
      train_op=train)

estimator = tf.contrib.learn.Estimator(model_fn=model)
# define our data sets
x_train = np.array([1., 2., 3., 4.])
y_train = np.array([0., -1., -2., -3.])
x_eval = np.array([2., 5., 8., 1.])
y_eval = np.array([-1.01, -4.1, -7, 0.])
input_fn = tf.contrib.learn.io.numpy_input_fn({"x": x_train}, y_train, 4, num_epochs=1000)
eval_input_fn = tf.contrib.learn.io.numpy_input_fn(
    {"x":x_eval}, y_eval, batch_size=4, num_epochs=1000)

# train
estimator.fit(input_fn=input_fn, steps=1000)
# Here we evaluate how well our model did.
train_loss = estimator.evaluate(input_fn=input_fn)
eval_loss = estimator.evaluate(input_fn=eval_input_fn)
print("train loss: %r"% train_loss)
print("eval loss: %r"% eval_loss)

代码分析

官方代码中，eval_input_fn缺少定义，请参考stackoverflow

于是，自己定义了eval_input_fn函数

eval_input_fn = tf.contrib.learn.io.numpy_input_fn(
    {"x":x_eval}, y_eval, batch_size=4, num_epochs=1000)

运行结果

官网提供

train loss: {'global_step': 1000, 'loss': 4.9380226e-11}
eval loss: {'global_step': 1000, 'loss': 0.01010081}

自己运行结果

train loss: {'global_step': 1000, 'loss': 2.8277852e-11}
eval loss: {'global_step': 1000, 'loss': 0.010100801}