[深度学习] (2):实现简单的线性回归(使用TensorFlow优化)

摘要

在学习使用TensorFlow, 动手写了LR的一般实现和TF实现，用的都是GD优化，但是TF明显快啊

PS: TF绘图的时候还加入了动画效果23333 _(:з」∠)_

---

普通实现

import numpy as np
import matplotlib.pyplot as plt

NUM_OF_POINTS = 100
NOISE_SHIFT = 20
N_ITER = 3000
alpha = 0.001

X = np.linspace(-10, 10, NUM_OF_POINTS)
y = 2 * X + np.random.rand(NUM_OF_POINTS) * NOISE_SHIFT
W = np.array([1])
bias = 0

cost_ = []
for i in range(N_ITER):
    error = X * W + bias - y
    cost = np.sum(np.square(error)) / (2 * NUM_OF_POINTS)
    cost_.append(cost)
    W = W - alpha * np.sum(error * X) / NUM_OF_POINTS
    bias = bias - alpha * np.sum(error) / NUM_OF_POINTS

# plot the result
fig = plt.figure(figsize=(14,5))
ax1 = fig.add_subplot(1,2,1)
ax1.scatter(X, y, c='r')
ax1.plot(X, X*W+bias, c='b')
ax2 = fig.add_subplot(1,2,2)
ax2.scatter(range(len(cost_)), cost_)
plt.show()

结果如下

---

TensorFlow实现 (绘图的时候还加入了动画效果 : )

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt

# 演示一个简单的LR回归

NUM_OF_POINTS = 100
NOISE_SHIFT = 20
N_ITER = 200
ALPHA = tf.constant(0.0001)

X_train = np.linspace(-10, 10, NUM_OF_POINTS)
y_train = 2 * X_train + np.random.rand(NUM_OF_POINTS) * NOISE_SHIFT


# Model parameters
W = tf.Variable([.3], dtype=tf.float32)
b = tf.Variable([-.3], dtype=tf.float32)
# Model input and output
x = tf.placeholder(tf.float32)
linear_model = W * x + b
y = tf.placeholder(tf.float32)

# loss
loss = tf.reduce_sum(tf.square(linear_model - y)) # sum of the squares
# optimizer
optimizer = tf.train.GradientDescentOptimizer(ALPHA)
train = optimizer.minimize(loss)


# plot the training data
fig = plt.figure(figsize=(14,5))
ax = fig.add_subplot(1,2,1)
ax.scatter(X_train, y_train)
ax_r = fig.add_subplot(1,2,2)
plt.ion()
plt.show()


with tf.Session() as sess:
    # 对于Variable必须先init才可以使用
    init = tf.global_variables_initializer()
    sess.run(init)
    loss_ = []
    for i in range(N_ITER):
        print(i)
        # 去除红线
        try:
            ax.lines.remove(lines[0])
        except Exception:
            pass


        sess.run(train, {x: X_train, y: y_train})
        loss_.append(sess.run(loss, {x: X_train, y: y_train}))

        prediction = sess.run(linear_model, feed_dict={x: X_train})
        lines = ax.plot(X_train, prediction, 'r-', lw=5)
        ax_r.scatter(i, sess.run(loss, {x: X_train, y: y_train}))
        plt.pause(0.1)

    print(sess.run([W, b]))

# plt.close()
# plt.scatter(range(len(loss_)), loss_)
# plt.show()
plt.pause(111)

---

总结

明显看到TF的GD比自己写的快了不少，在100次左右就几乎收敛了，而且步长还小，不知道底层是怎么优化的，直接解正规方程吗2333，还是用牛顿法优化2333，都不会思密达，继续学习，怀挺！