Getting Started with TensorFlow

TensorFlow provides multiple APIs. The lowest level API TensorFlow Core provides you with complete programming control. (But it’s difficult to start with) The higher level APIs, which are bulit on top of TensorFlow Core, are typically easier to learn and use.

---

1. Basic steps

The central unit of data in TensorFlow is the tensor. (n-rank: n indexes for one component)

# gives Python access all the tensorFlow's classes
import tensorflow as tf  

The Computational Graph

1. Building the computational graph (create a graph)
2. Running the computational graph (launch it in a session)

A computational graph is a series of TensorFlow operations arranged into a graph of nodes. Each node takes zero or more tensors as inputs and produces one tensor as an output.

1.1 Create nodes

 # constant node.(there are other nodes)
 node1 = tf.constant(3.0, tf.float32)

only when evaluated, would produce 3

---

1.2 Evaluate nodes

To actually evaluate the nodes, we must runt the computational graph with session, which encapsulates the control and state of its runtime.

sess = tf.Session()
print(sess.run([node1, node2]))

---

1.3 Building more complicated computation

By incorporating Tensor nodes with operations (Operations are also nodes)

node3 = tf.add(node1, node2)
print("node3: ", node3)
print("sess.run(node3): ",sess.run(node3))

TensorFlow provides a utility called TensorBoard that can display a picture of the computational graph

---

1.4 Using placeholder to accept external inputs

a = tf.placeholder(tf.float32)
b = tf.placeholder(tf.float32)
adder_node = a + b  # + provides a shortcut for tf.add(a, b)

The placeholder node can be fed with multiple inputs by python dict with {node_name: list}

print(sess.run(adder_node, {a: 3, b:4.5}))  # result: 7.5
print(sess.run(adder_node, {a: [1,3], b: [2, 4]})) # result: [3,7]

---

1.5 Using Variable node to add trainable parameters (w,b)

W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
x = tf.placeholder(tf.float32)
linear_model = W * x + b

Variable node has a type and initial value.

Constants are initialized when you call tf.constant, and their value can never change. By contrast, variables are not initialized when you call tf.Variable. To initialize all the variables in a TensorFlow program, you must explicitly call a special operation as follows:

init = tf.global_variables_initializer()
sess.run(init)

---

1.6 Using y placeholder as desired values to evaluate

# square cost function
y = tf.placeholder(tf.float32)
squared_deltas = tf.square(linear_model - y)
loss = tf.reduce_sum(squared_deltas)
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))

We could improve performance (decrease cost) by reassigning the parameters (w,b), which are Variable nodes:
A variable is initialized to the value provided to tf.Variable but can be changed using operations like tf.assign.

fixW = tf.assign(W, [-1.])
fixb = tf.assign(b, [1.])
sess.run([fixW, fixb]) #update the W and b nodes
print(sess.run(loss, {x:[1,2,3,4], y:[0,-1,-2,-3]}))

---

1.7 Using tf.train API to find optimal (W, b) automatically

TensorFlow provides optimizers that slowly change each variable in order to minimize the loss function. The simplest optimizer is gradient descent.

Besides, tf.gradients can be used to produce derivatives.

For example,

# create optimizer with learning rate(0.01)
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)

sess.run(init) # reset values to incorrect defaults.
for i in range(1000):
  sess.run(train, {x:[1,2,3,4], y:[0,-1,-2,-3]})

print(sess.run([W, b]))

A standard sample:

import numpy as np
import tensorflow as tf

# Model parameters
W = tf.Variable([.3], tf.float32)
b = tf.Variable([-.3], tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W * x + b
y = tf.placeholder(tf.float32)
# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
optimizer = tf.train.GradientDescentOptimizer(0.01)
train = optimizer.minimize(loss)
# training data
x_train = [1,2,3,4]
y_train = [0,-1,-2,-3]
# training loop
init = tf.global_variables_initializer()
sess = tf.Session()
sess.run(init) # reset values to wrong
for i in range(1000):
  sess.run(train, {x:x_train, y:y_train})

# evaluate training accuracy
curr_W, curr_b, curr_loss  = sess.run([W, b, loss], {x:x_train, y:y_train})
print("W: %s b: %s loss: %s"%(curr_W, curr_b, curr_loss))

2. Introduction to tf.contrib.learn

tf.contrib.learn is a high-level TensorFlow library that simplifies the mechanics of machine learning, including:

• running training loops
• running evaluation loops
• managing data sets
• managing feeding

Basic usage of tf.contrib.learn

1.declare list of features
2.construct estimator
3.Using helper methods to read and set up data sets (x, y) input_fn
4. invoke fit method to indicate the training steps
5. evaluate the mode estimator.evaluate(input_fn=input_fn)

import tensorflow as tf
# NumPy is often used to load, manipulate and preprocess data.
import numpy as np

# Declare list of features. We only have one real-valued feature. There are many
# other types of columns that are more complicated and useful.
features = [tf.contrib.layers.real_valued_column("x", dimension=1)]

# An estimator is the front end to invoke training (fitting) and evaluation
# (inference). There are many predefined types like linear regression,
# logistic regression, linear classification, logistic classification, and
# many neural network classifiers and regressors. The following code
# provides an estimator that does linear regression.
estimator = tf.contrib.learn.LinearRegressor(feature_columns=features)

# TensorFlow provides many helper methods to read and set up data sets.
# Here we use `numpy_input_fn`. We have to tell the function how many batches
# of data (num_epochs) we want and how big each batch should be.
x = np.array([1., 2., 3., 4.])
y = np.array([0., -1., -2., -3.])
input_fn = tf.contrib.learn.io.numpy_input_fn({"x":x}, y, batch_size=4,
                                              num_epochs=1000)

# We can invoke 1000 training steps by invoking the `fit` method and passing the
# training data set.
estimator.fit(input_fn=input_fn, steps=1000)

# Here we evaluate how well our model did. In a real example, we would want
# to use a separate validation and testing data set to avoid overfitting.
print(estimator.evaluate(input_fn=input_fn))

3. Build a custom model

To define a custom model that works with tf.contrib.learn, we need to use tf.contrib.learn.Estimator. tf.contrib.learn.LinearRegressor is actually a sub-class of tf.contrib.learn.Estimator. Instead of sub-classing Estimator, we simply provide Estimator a function model_fn (function parameter) that tells tf.contrib.learn how it can evaluate predictions, training steps, and loss. The code is as follows:

import numpy as np
import tensorflow as tf
# Declare list of features, we only have one real-valued feature
def model(features, labels, mode):
  # Build a linear model and predict values
  W = tf.get_variable("W", [1], dtype=tf.float64)
  b = tf.get_variable("b", [1], dtype=tf.float64)
  y = W*features['x'] + b
  # Loss sub-graph
  loss = tf.reduce_sum(tf.square(y - labels))
  # Training sub-graph
  global_step = tf.train.get_global_step()
  optimizer = tf.train.GradientDescentOptimizer(0.01)
  train = tf.group(optimizer.minimize(loss),
                   tf.assign_add(global_step, 1))
  # ModelFnOps connects subgraphs we built to the
  # appropriate functionality.
  return tf.contrib.learn.ModelFnOps(
      mode=mode, predictions=y,
      loss=loss,
      train_op=train)

estimator = tf.contrib.learn.Estimator(model_fn=model)
# define our data set
x = np.array([1., 2., 3., 4.])
y = np.array([0., -1., -2., -3.])
input_fn = tf.contrib.learn.io.numpy_input_fn({"x": x}, y, 4, num_epochs=1000)

# train
estimator.fit(input_fn=input_fn, steps=1000)
# evaluate our model
print(estimator.evaluate(input_fn=input_fn, steps=10))