文章标题

Logistic Regression

• 代码
• 要点
• 总结

---

代码

import numpy as np
import tensorflow as tf

# Import MINST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# input and model parameters
# input: type and size
X = tf.placeholder(tf.float32, [None, 784])
Y = tf.placeholder(tf.float32, [None, 10])

# parameters: size with type, name
W = tf.Variable(tf.zeros([784, 10]), name="weight")
b = tf.Variable(tf.zeros([10]), name = "bias")

# hyperparameters
learning_rate = 0.01
training_epochs = 25
batch_size = 100
display_step = 1

# make prediction, pred: [None, 10]
pred = tf.nn.softmax(tf.matmul(X, W) + b)

# cost function using cross entropy
# using tf.reduce_mean coz we don't know the number of samples for now
# reduction_indices = 1 coz we sum by each sample (ten to one)
# after the summation, data should be [None, 1]
cost = tf.reduce_mean(-tf.reduce_sum(Y*tf.log(pred), reduction_indices=1))
# recall reduction_indices=1 means summing in row order

trainer = tf.train.GradientDescentOptimizer(learning_rate=learning_rate)

optimizer = trainer.minimize(cost)

init = tf.global_variables_initializer()

with tf.Session() as sess:
    sess.run(init)

    for epoch in range(training_epochs):
        ave_cost = 0
        total_batch = int(mnist.train.num_examples / batch_size)

        for batch in range(total_batch):
            batch_xs, batch_ys = mnist.train.next_batch(batch_size)
            # run two operations together with same input set, use _ to omit the returned result
            _, c = sess.run([optimizer, cost], feed_dict={X: batch_xs, Y: batch_ys})
            ave_cost += c / total_batch

        if (epoch+1) % display_step == 0:
            # recall the use of "{:.9f}".format(...)
            print("epoch %2d" % (epoch+1), "average cost: ", "{:.9f}".format(ave_cost))

---

要点

• cost的计算使用了tf.reduce_mean套用tf.reduce_sum，和之前tf.reduce_sum / n_samples用意是一样的，只是我们现在sample数量可变（未知），所以用tf_reduce_mean求均值
• 复习tf.Variable和tf.placeholder的使用。tf.placeholder(tf.float32, [None, 784]，其中None代表根据运行变化。tf.Variable(tf.zeros([784, 10]))，name可有可无，不能在第一个参数只用[784, 10]，需要通过tf.zeros来确定该变量的类型（float32），从而与placeholder对应。

---

总结

此段代码为sigmoid作为binary softmax的特例，nothing special。