我的tensorflow学习笔记（3）

一、小技巧

1、查看tensorflow安装路径

import tensorflow as tf
tf.__path__ #查看路径
tf.__version__ #查看版本

二、linear regression

1、源码来自stanford-tensorflow-tutorials 03_linreg

2、在自己写的时候发现一个问题，即tf.add与+的区别。百度后的结果：

    tf.add(a,b)与a+b没有精度区别，在计算精度上，二者并没有差别。运算符重载的形式a+b，会在内部转换为，a.__add__(b)，而a.__add__(b)会再一次地映射为tf.add，在 math_ops.py中相关的映射如下：

_OverrideBinaryOperatorHelper(gen_math_ops.add, "add")

    参考链接：  In tensorflow what is the difference between tf.add and operator (+)?

3、关键代码        

# Step 6: using gradient descent with learning rate of 0.001 to minimize loss
optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)

# Step 8: train the model for 100 epochs
	for i in range(100): 
		total_loss = 0
		for x, y in data:
			# Session execute optimizer and fetch values of loss
			_, l = sess.run([optimizer, loss], feed_dict={X: x, Y:y}) 
			total_loss += l
		print('Epoch {0}: {1}'.format(i, total_loss/n_samples))

4、hubber loss

    使用hubber loss来减少离群值的weight：

    在tensorflow中，涉及图的控制流不能使用简单的If-else语句（这只在eager execution中可行），而应使用tf.cond：

tf.cond(pred, fn1, fn2, name=None)

    format：tf.cond(pred, fn1, fn2, name=None)
    Return :either fn1() or fn2() based on the boolean predicate `pred`.(注意这里，也就是说'fnq'和‘fn2’是两个函数)
   arguments:`fn1` and `fn2` both return lists of output tensors. `fn1` and `fn2` must have the same non-zero number and type of outputs('fnq'和‘fn2’返回的是非零的且类型相同的输出)

def huber_loss(labels, predictions, delta=14.0):
    residual = tf.abs(labels - predictions)
    def f1(): return 0.5 * tf.square(residual)
    def f2(): return delta * residual - 0.5 * tf.square(delta)
    return tf.cond(residual < delta, f1, f2)

5、tf.data

   使用tf.data而非placeholder

# Step 1: read in the data
data, n_samples = utils.read_birth_life_data(DATA_FILE)

# Step 2: create Dataset and iterator
dataset = tf.data.Dataset.from_tensor_slices((data[:,0], data[:,1]))

iterator = dataset.make_initializable_iterator()
X, Y = iterator.get_next()

# Step 8: train the model for 100 epochs
    for i in range(100):
        sess.run(iterator.initializer) # initialize the iterator
        total_loss = 0
        try:
            while True:
                _, l = sess.run([optimizer, loss]) 
                total_loss += l
        except tf.errors.OutOfRangeError:
            pass
            
        print('Epoch {0}: {1}'.format(i, total_loss/n_samples))

    有两种iterator：

iterator = dataset.make_one_shot_iterator()
#Iterates through the dataset exactly once. No need to initialization.
iterator = dataset.make_initializable_iterator()
#Iterates through the dataset as many times as we want. Need to initialize with each epoch.

    其中make_one_shot_iterator()无需初始化，但只能从头到尾读取一次；而make_initializable_iterator()可以读取任意次，但每次重新读取都要进行初始化。

6、optimizer

optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(loss)
sess.run([optimizer]) 

    采用梯度下降法，最小化loss。默认的，optimizer会训练与其目标函数有关的所有可训练的变量。如果有不想训练的对象，需要加上关键词：

global_step = tf.Variable(0, trainable=False, dtype=tf.int32)