tensorflow的examples下Udacity中Assignment 2的problem解决方案

抽空看了下udacity上的deeplearning课程，讲的不错。尝试着做里面的题目，csdn上也有很多Assignment的教程，我也把我写的添加进来，供大家参考：

一、问题重述：

Problem

 Turn the logistic regression example with SGD into a 1-hidden layer neural network with rectified linear units nn.relu() and 1024 hidden nodes. This model should improve your validation / test accuracy.

意思大概是让我们把只有一层神经网络的基于随机梯度下降的逻辑回归通过nn.relu()，加上一个隐藏层。隐藏层的神经单元的个数设置为1024。

我所写的代码跟其他人不同的地方在于很多教程都是直接在原有的代码基础上添加了一个隐藏层，代码虽然没有错误，但是结构不是很明确，因此，我把weight和baies提取出来，并且将添加层数封装成一个函数。部分代码如下：

hidden_units = 1024
batch_size = 128

graph = tf.Graph()
with graph.as_default():

    with tf.name_scope('weight'):
        weight = {
            "w1": tf.Variable(tf.truncated_normal([image_size*image_size, hidden_units])),
            "w2": tf.Variable(tf.truncated_normal([hidden_units, num_labels]))
        }
    with tf.name_scope('baies'):
        baies = {
            "b1": tf.Variable(tf.zeros([hidden_units])),
            "b2": tf.Variable(tf.zeros([num_labels]))
        }


    def multi_layers(input_data, weight, baies):
        with tf.name_scope('layer_1'):
            logits_1 = tf.matmul(input_data, weight['w1']) + baies['b1']
        with tf.name_scope('relu'):
            hidden_layer = tf.nn.relu(logits_1, name='hidden_layer')
        with tf.name_scope('layer_2'):
            logits_2 = tf.matmul(hidden_layer, weight['w2']) + baies['b2']

        return logits_2

    with tf.name_scope('input_data'):
        with tf.name_scope('train_data'):
            tf_train_data = tf.placeholder(tf.float32, shape=(batch_size, image_size * image_size))
        with tf.name_scope('train_labels'):
            tf_train_labels = tf.placeholder(tf.float32, shape=(batch_size, num_labels))
        with tf.name_scope('valid_data'):
            tf_valid_data = tf.constant(valid_dataset)
        with tf.name_scope('test_data'):
            tf_test_data = tf.constant(test_dataset)

    with tf.name_scope('loss'):
        predict = multi_layers(tf_train_data, weight, baies)
        loss = tf.reduce_mean(
            tf.nn.softmax_cross_entropy_with_logits(labels=tf_train_labels, logits=predict, name='loss'))

    with tf.name_scope('optimizer'):
        optimizer = tf.train.GradientDescentOptimizer(0.5).minimize(loss)

    with tf.name_scope('train_prediction'):
        train_prediction = tf.nn.softmax(predict)
    with tf.name_scope('valid_prediction'):
        valid_predict = multi_layers(valid_dataset, weight, baies)
        valid_prediction = tf.nn.softmax(valid_predict)
    with tf.name_scope('test_prediction'):
        test_predict = multi_layers(test_dataset, weight, baies)
        test_prediction = tf.nn.softmax(test_predict)

num_steps = 3001

with tf.Session(graph=graph) as session:
    tf.global_variables_initializer().run()
    writer = tf.summary.FileWriter("logss/", session.graph)
    saver = tf.train.Saver()

    print("Initialized")
    for step in range(num_steps):
        # Pick an offset within the training data, which has been randomized.
        #  Note: we could use better randomization across epochs.
        offset = (step * batch_size) % (train_labels.shape[0] - batch_size)
        # Generate a minibatch.
        batch_data = train_dataset[offset:(offset + batch_size), :]
        batch_labels = train_labels[offset:(offset + batch_size), :]
        # Prepare a dictionary telling the session where to feed the minibatch.
        # The key of the dictionary is the placeholder node of the graph to be fed,
        # and the value is the numpy array to feed to it.
        feed_dict = {tf_train_data: batch_data, tf_train_labels: batch_labels}
        _, l, predictions = session.run(
            [optimizer, loss, train_prediction], feed_dict=feed_dict)

        if step % 500 == 0:
            print("Minibatch loss at step %d: %f" % (step, l))
            print("Minibatch accuracy: %.1f%%" % accuracy(predictions, batch_labels))
            print("Validation accuracy: %.1f%%" % accuracy(
                valid_prediction.eval(), valid_labels))
    print("Test accuracy: %.1f%%" % accuracy(test_prediction.eval(), test_labels))

代码中添加了graph name 因此可以在tensorboard中查看tensor的处理过程。如下图

从图中可以看出train,valid,test_data共用同一对weight和baies。train_data的训练过程等。当然我们也可以通过tensorboard来检测自己的网络搭建过程中的错误。

二、运行结果：

添加了一层隐藏层之后，结果却是提高了点，但不是很显著，因此可以继续优化。

Initialized
Minibatch loss at step 0: 308.016052
Minibatch accuracy: 9.4%
Validation accuracy: 28.5%
Minibatch loss at step 500: 22.196594
Minibatch accuracy: 81.2%
Validation accuracy: 79.3%
Minibatch loss at step 1000: 10.916459
Minibatch accuracy: 82.0%
Validation accuracy: 80.5%
Minibatch loss at step 1500: 6.220052
Minibatch accuracy: 88.3%
Validation accuracy: 80.7%
Minibatch loss at step 2000: 2.950284
Minibatch accuracy: 86.7%
Validation accuracy: 82.2%
Minibatch loss at step 2500: 2.925977
Minibatch accuracy: 86.7%
Validation accuracy: 81.7%
Minibatch loss at step 3000: 2.005749
Minibatch accuracy: 82.0%
Validation accuracy: 81.8%
Test accuracy: 89.2%