8 卷积神经网络——解决参数太多问题（1）

最新推荐文章于 2021-10-17 21:50:58 发布
最新推荐文章于 2021-10-17 21:50:58 发布
阅读量1k 收藏 1
点赞数
分类专栏： 深度学习之TensorFlow入门、原理与进阶实战

8-6 cifar手动读取

程序：

import numpy as np
from scipy.misc import imsave

filename = 'D:/temp/cifar10_data/cifar-10-batches-bin/test_batch.bin'

bytestream = open(filename, "rb")
buf = bytestream.read(10000 * (1 + 32 * 32 * 3))
bytestream.close()

data = np.frombuffer(buf, dtype=np.uint8)
data = data.reshape(10000, 1 + 32 * 32 * 3)
labels_images = np.hsplit(data, [1])
labels = labels_images[0].reshape(10000)
images = labels_images[1].reshape(10000, 32, 32, 3)

img = np.reshape(images[0], (3, 32, 32))  # 导出第一幅图
img = img.transpose(1, 2, 0)

import pylab

print(labels[0])
pylab.imshow(img)
pylab.show()

结果：

在这里插入图片描述
在这里插入图片描述

8-7 queue

程序：

import tensorflow as tf

# 创建长度为100的队列
queue = tf.FIFOQueue(100, "float")

c = tf.Variable(0.0)  # 计数器
# 加1操作
op = tf.assign_add(c, tf.constant(1.0))
# 操作:将计数器的结果加入队列
enqueue_op = queue.enqueue(c)

# 创建一个队列管理器QueueRunner，用这两个操作向q中添加元素。目前我们只使用一个线程:
qr = tf.train.QueueRunner(queue, enqueue_ops=[op, enqueue_op])

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    coord = tf.train.Coordinator()

    ## 启动入队线程, Coordinator是线程的参数
    enqueue_threads = qr.create_threads(sess, coord=coord, start=True)  # 启动入队线程

    # 主线程
    for i in range(0, 10):
        print("-------------------------")
        print(sess.run(queue.dequeue()))

    coord.request_stop()  # 通知其他线程关闭 其他所有线程关闭之后，这一函数才能返回

    # join操作经常用在线程当中,其作用是等待某线程结束
    # coord.join(enqueue_threads)

结果：
在这里插入图片描述
8-10 MNIST卷积
程序：

import tensorflow as tf
# 导入 MINST 数据集
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("F:/shendu/MNIST_data/", one_hot=True)

def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)

def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)

def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')

def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                          strides=[1, 2, 2, 1], padding='SAME')

def avg_pool_7x7(x):
    return tf.nn.avg_pool(x, ksize=[1, 7, 7, 1],
                          strides=[1, 7, 7, 1], padding='SAME')

# tf Graph Input
x = tf.placeholder(tf.float32, [None, 784])  # mnist data维度 28*28=784
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classes

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

x_image = tf.reshape(x, [-1, 28, 28, 1])

h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)

W_conv2 = weight_variable([5, 5, 32, 64])
b_conv2 = bias_variable([64])

h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)
#########################################################new
W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)

nt_hpool3 = avg_pool_7x7(h_conv3)  # 64
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)

cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

# 启动session
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(200):  # 20000
        batch = mnist.train.next_batch(50)  # 50
        if i % 20 == 0:
            train_accuracy = accuracy.eval(feed_dict={
                x: batch[0], y: batch[1]})
            print("step %d, training accuracy %g" % (i, train_accuracy))
        train_step.run(feed_dict={x: batch[0], y: batch[1]})

    print("test accuracy %g" % accuracy.eval(feed_dict={
        x: mnist.test.images, y: mnist.test.labels}))

结果：

Extracting F:/shendu/MNIST_data/train-images-idx3-ubyte.gz

Extracting F:/shendu/MNIST_data/train-labels-idx1-ubyte.gz

Extracting F:/shendu/MNIST_data/t10k-images-idx3-ubyte.gz
Extracting F:/shendu/MNIST_data/t10k-labels-idx1-ubyte.gz

step 0, training accuracy 0.1
step 20, training accuracy 0.06
step 40, training accuracy 0.06
step 60, training accuracy 0.1
step 80, training accuracy 0.16
step 100, training accuracy 0.2
step 120, training accuracy 0.12
step 140, training accuracy 0.18
step 160, training accuracy 0.26
step 180, training accuracy 0.26
test accuracy 0.3052

8-12反卷积操作
通过模拟数据进行卷积和反卷积操作，比较卷积与反卷积中padding在SAME，VALID下的变化，

程序：

import numpy as np
import tensorflow as tf

img = tf.Variable(tf.constant(1.0, shape=[1, 4, 4, 1]))#模拟数据

filter = tf.Variable(tf.constant([1.0, 0, -1, -2], shape=[2, 2, 1, 1]))

conv = tf.nn.conv2d(img, filter, strides=[1, 2, 2, 1], padding='VALID')#分别进行VALID和SAME操作
cons = tf.nn.conv2d(img, filter, strides=[1, 2, 2, 1], padding='SAME')
print(conv.shape)
print(cons.shape)

contv = tf.nn.conv2d_transpose(conv, filter, [1, 4, 4, 1], strides=[1, 2, 2, 1], padding='VALID')#反卷积
conts = tf.nn.conv2d_transpose(cons, filter, [1, 4, 4, 1], strides=[1, 2, 2, 1], padding='SAME')

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    print("conv:\n", sess.run([conv, filter]))
    print("cons:\n", sess.run([cons]))
    print("contv:\n", sess.run([contv]))
    print("conts:\n", sess.run([conts]))

结果：

(1, 2, 2, 1)
(1, 2, 2, 1)

conv:
 [array([[[[-2.],
         [-2.]],

        [[-2.],
         [-2.]]]], dtype=float32), array([[[[ 1.]],

        [[ 0.]]],


       [[[-1.]],

        [[-2.]]]], dtype=float32)]
cons:
 [array([[[[-2.],
         [-2.]],

        [[-2.],
         [-2.]]]], dtype=float32)]
contv:
 [array([[[[-2.],
         [ 0.],
         [-2.],
         [ 0.]],

        [[ 2.],
         [ 4.],
         [ 2.],
         [ 4.]],

        [[-2.],
         [ 0.],
         [-2.],
         [ 0.]],

        [[ 2.],
         [ 4.],
         [ 2.],
         [ 4.]]]], dtype=float32)]
conts:
 [array([[[[-2.],
         [ 0.],
         [-2.],
         [ 0.]],

        [[ 2.],
         [ 4.],
         [ 2.],
         [ 4.]],

        [[-2.],
         [ 0.],
         [-2.],
         [ 0.]],

        [[ 2.],
         [ 4.],
         [ 2.],
         [ 4.]]]], dtype=float32)]

8-13 反池化操作

定义一个数组作为模拟图片，将其进行最大池化，接着再进行反池化，比较原始数据与反池化后的数据。

程序：

import tensorflow as tf
import numpy as np


def max_pool_with_argmax(net, stride):#重新定义最大池化函数
    _, mask = tf.nn.max_pool_with_argmax(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1],
                                         padding='SAME')
    mask = tf.stop_gradient(mask)
    net = tf.nn.max_pool(net, ksize=[1, stride, stride, 1], strides=[1, stride, stride, 1], padding='SAME')
    return net, mask

def unpool(net, mask, stride):#定义反最大池化
    ksize = [1, stride, stride, 1]
    input_shape = net.get_shape().as_list()
    #  calculation new shape
    output_shape = (input_shape[0], input_shape[1] * ksize[1], input_shape[2] * ksize[2], input_shape[3])
    #计算索引 calculation indices for batch, height, width and feature maps
    one_like_mask = tf.ones_like(mask)
    batch_range = tf.reshape(tf.range(output_shape[0], dtype=tf.int64), shape=[input_shape[0], 1, 1, 1])
    b = one_like_mask * batch_range
    y = mask // (output_shape[2] * output_shape[3])
    x = mask % (output_shape[2] * output_shape[3]) // output_shape[3]
    feature_range = tf.range(output_shape[3], dtype=tf.int64)
    f = one_like_mask * feature_range
    #转置索引 transpose indices & reshape update values to one dimension
    updates_size = tf.size(net)
    indices = tf.transpose(tf.reshape(tf.stack([b, y, x, f]), [4, updates_size]))
    values = tf.reshape(net, [updates_size])
    ret = tf.scatter_nd(indices, values, output_shape)
    return ret


img = tf.constant([
    [[0.0, 4.0], [0.0, 4.0], [0.0, 4.0], [0.0, 4.0]],
    [[1.0, 5.0], [1.0, 5.0], [1.0, 5.0], [1.0, 5.0]],
    [[2.0, 6.0], [2.0, 6.0], [2.0, 6.0], [2.0, 6.0]],
    [[3.0, 7.0], [3.0, 7.0], [3.0, 7.0], [3.0, 7.0]]
])#定义数组

img = tf.reshape(img, [1, 4, 4, 2])
pooling2 = tf.nn.max_pool(img, [1, 2, 2, 1], [1, 2, 2, 1], padding='SAME')#池化
encode, mask = max_pool_with_argmax(img, 2)


img2 = unpool(encode, mask, 2)#反池化
print(img.shape)
print(encode.shape)
print(mask.shape)
print(img2.shape)
with tf.Session() as sess:
    print("image:")
    print(sess.run(img))
    result = sess.run(pooling2)
    print("pooling2:\n", result)
    result, mask2 = sess.run([encode, mask])
    print("encode:\n", result, mask2)

    print("-------------------------------")
    result = sess.run(img2)
    print("reslut:\n", result)

结果：

(1, 4, 4, 2)
(1, 2, 2, 2)

(1, 4, 4, 2)

image:

[[[[0. 4.]
   [0. 4.]
   [0. 4.]
   [0. 4.]]

  [[1. 5.]
   [1. 5.]
   [1. 5.]
   [1. 5.]]

  [[2. 6.]
   [2. 6.]
   [2. 6.]
   [2. 6.]]

  [[3. 7.]
   [3. 7.]
   [3. 7.]
   [3. 7.]]]]
pooling2:
 [[[[1. 5.]
   [1. 5.]]

  [[3. 7.]
   [3. 7.]]]]
encode:
 [[[[1. 5.]
   [1. 5.]]

  [[3. 7.]
   [3. 7.]]]] [[[[ 8  9]
   [12 13]]

  [[24 25]
   [28 29]]]]
-------------------------------
reslut:
 [[[[0. 0.]
   [0. 0.]
   [0. 0.]
   [0. 0.]]

  [[1. 5.]
   [0. 0.]
   [1. 5.]
   [0. 0.]]

  [[0. 0.]
   [0. 0.]
   [0. 0.]
   [0. 0.]]

  [[3. 7.]
   [0. 0.]
   [3. 7.]
   [0. 0.]]]]

8-15 gradients1

有2个op，4个参数，演示使用gradients同时为两个式子4个参数求梯度。

程序：

import tensorflow as tf

tf.reset_default_graph()
w1 = tf.get_variable('w1', shape=[2])
w2 = tf.get_variable('w2', shape=[2])

w3 = tf.get_variable('w3', shape=[2])
w4 = tf.get_variable('w4', shape=[2])

y1 = w1 + w2+ w3
y2 = w3 + w4

a = w1+w2
a_stoped = tf.stop_gradient(a)
y3= a_stoped+w3

gradients = tf.gradients([y1, y2], [w1, w2, w3, w4], grad_ys=[tf.convert_to_tensor([1.,2.]),
                                                          tf.convert_to_tensor([3.,4.])])#grad_ys求梯度的输入值
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(gradients))

结果：

[array([1., 2.], dtype=float32), array([1., 2.], dtype=float32), array([4., 6.], dtype=float32), array([3., 4.], dtype=float32)]

8-16 gradients2

演示梯度停止的用法，并观察当变量设置梯度停止后，对其求梯度的结果。

程序：

import tensorflow as tf

tf.reset_default_graph()
w1 = tf.get_variable('w1', shape=[2])
w2 = tf.get_variable('w2', shape=[2])

w3 = tf.get_variable('w3', shape=[2])
w4 = tf.get_variable('w4', shape=[2])

y1 = w1 + w2 + w3
y2 = w3 + w4

a = w1 + w2
a_stoped = tf.stop_gradient(a)
y3 = a_stoped + w3

gradients = tf.gradients([y1, y2], [w1, w2, w3, w4], grad_ys=[tf.convert_to_tensor([1., 2.]),
                                                              tf.convert_to_tensor([3., 4.])])

gradients2 = tf.gradients(y3, [w1, w2, w3], grad_ys=tf.convert_to_tensor([1., 2.]))
print(gradients2)

gradients3 = tf.gradients(y3, [w3], grad_ys=tf.convert_to_tensor([1., 2.]))

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    print(sess.run(gradients))
    # print(sess.run(gradients2))#程序试图求一个None的梯度，报错
    print(sess.run(gradients3))

结果：

[None, None, <tf.Tensor 'gradients_1/grad_ys_0:0' shape=(2,) dtype=float32>]

[array([1., 2.], dtype=float32), array([1., 2.], dtype=float32), array([4., 6.], dtype=float32), array([3., 4.], dtype=float32)]
[array([1., 2.], dtype=float32)]

8-22 带BN的多通道MNIST

程序：

import tensorflow as tf
import numpy as np
from tensorflow.contrib.layers.python.layers import batch_norm
# 导入 MINST 数据集
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("F:/shendu/MNIST_data/", one_hot=True)


def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)


def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)


def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')


def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                          strides=[1, 2, 2, 1], padding='SAME')


def avg_pool_7x7(x):
    return tf.nn.avg_pool(x, ksize=[1, 7, 7, 1],
                          strides=[1, 7, 7, 1], padding='SAME')


def batch_norm_layer(value, train=None, name='batch_norm'):#BN函数
    if train is not None:
        return batch_norm(value, decay=0.9, updates_collections=None, is_training=True)
    else:
        return batch_norm(value, decay=0.9, updates_collections=None, is_training=False)


# tf Graph Input
x = tf.placeholder(tf.float32, [None, 784])  # mnist data维度 28*28=784
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classes
train = tf.placeholder(tf.float32)

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

x_image = tf.reshape(x, [-1, 28, 28, 1])
h_conv1 = tf.nn.relu(batch_norm_layer((conv2d(x_image, W_conv1) + b_conv1), train))
h_pool1 = max_pool_2x2(h_conv1)
######################################################多卷积核
W_conv2_5x5 = weight_variable([5, 5, 32, 32])
b_conv2_5x5 = bias_variable([32])
W_conv2_7x7 = weight_variable([7, 7, 32, 32])
b_conv2_7x7 = bias_variable([32])
h_conv2_5x5 = tf.nn.relu(batch_norm_layer((conv2d(h_pool1, W_conv2_5x5) + b_conv2_5x5), train))
h_conv2_7x7 = tf.nn.relu(batch_norm_layer((conv2d(h_pool1, W_conv2_7x7) + b_conv2_7x7), train))
h_conv2 = tf.concat([h_conv2_5x5, h_conv2_7x7], 3)

h_pool2 = max_pool_2x2(h_conv2)
#########################################################new 池化

W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)

nt_hpool3 = avg_pool_7x7(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)

keep_prob = tf.placeholder("float")

cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))

decaylearning_rate = tf.train.exponential_decay(0.04, 20000, 1000, 0.9)
train_step = tf.train.AdamOptimizer(decaylearning_rate).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

# 启动session
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(20000):  # 20000
        batch = mnist.train.next_batch(50)
        if i % 100 == 0:
            train_accuracy = accuracy.eval(feed_dict={
                x: batch[0], y: batch[1], keep_prob: 1.0})
            print("step %d, training accuracy %g" % (i, train_accuracy))
        train_step.run(feed_dict={x: batch[0], y: batch[1], keep_prob: 0.5})

    print("test accuracy %g" % accuracy.eval(feed_dict={
        x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}))

结果：

D:\Anaconda3\python.exe F:/PycharmProjects/test20190624/test.py

Extracting F:/shendu/MNIST_data/train-images-idx3-ubyte.gz

Extracting F:/shendu/MNIST_data/train-labels-idx1-ubyte.gz

Extracting F:/shendu/MNIST_data/t10k-images-idx3-ubyte.gz
Extracting F:/shendu/MNIST_data/t10k-labels-idx1-ubyte.gz

step 0, training accuracy 0.14
step 100, training accuracy 0.68
step 200, training accuracy 0.8
step 300, training accuracy 0.7
step 400, training accuracy 0.86
step 500, training accuracy 0.7
step 600, training accuracy 0.8
step 700, training accuracy 0.74
step 800, training accuracy 0.9
step 900, training accuracy 0.86
step 1000, training accuracy 0.78
step 1100, training accuracy 0.84
step 1200, training accuracy 0.84
step 1300, training accuracy 0.96
step 1400, training accuracy 0.84
step 1500, training accuracy 0.92
step 1600, training accuracy 0.9
step 1700, training accuracy 0.88
step 1800, training accuracy 0.88
step 1900, training accuracy 0.88
step 2000, training accuracy 0.86
step 2100, training accuracy 0.9
step 2200, training accuracy 0.82
step 2300, training accuracy 0.82
step 2400, training accuracy 0.86
step 2500, training accuracy 0.9
step 2600, training accuracy 0.88
step 2700, training accuracy 0.86
step 2800, training accuracy 0.96
step 2900, training accuracy 0.86
step 3000, training accuracy 0.86
step 3100, training accuracy 0.92
step 3200, training accuracy 0.96
step 3300, training accuracy 0.9
step 3400, training accuracy 0.96
step 3500, training accuracy 0.86
step 3600, training accuracy 0.86
step 3700, training accuracy 0.82
step 3800, training accuracy 0.86
step 3900, training accuracy 0.86
step 4000, training accuracy 0.88
step 4100, training accuracy 0.88
step 4200, training accuracy 0.94
step 4300, training accuracy 0.84
step 4400, training accuracy 0.86
step 4500, training accuracy 0.92
step 4600, training accuracy 0.9
step 4700, training accuracy 0.84
step 4800, training accuracy 0.86
step 4900, training accuracy 0.88
step 5000, training accuracy 0.88
step 5100, training accuracy 0.9
step 5200, training accuracy 0.88
step 5300, training accuracy 0.9
step 5400, training accuracy 0.9
step 5500, training accuracy 0.9
step 5600, training accuracy 0.88
step 5700, training accuracy 0.98
step 5800, training accuracy 0.98
step 5900, training accuracy 0.98
step 6000, training accuracy 1
step 6100, training accuracy 1
step 6200, training accuracy 0.98
step 6300, training accuracy 0.98
step 6400, training accuracy 0.98
step 6500, training accuracy 1
step 6600, training accuracy 1
step 6700, training accuracy 1
step 6800, training accuracy 1
step 6900, training accuracy 1
step 7000, training accuracy 1
step 7100, training accuracy 1
step 7200, training accuracy 1
step 7300, training accuracy 1
step 7400, training accuracy 1
step 7500, training accuracy 1
step 7600, training accuracy 1
step 7700, training accuracy 1
step 7800, training accuracy 0.98
step 7900, training accuracy 0.98
step 8000, training accuracy 1
step 8100, training accuracy 1
step 8200, training accuracy 1
step 8300, training accuracy 1
step 8400, training accuracy 1
step 8500, training accuracy 1
step 8600, training accuracy 1
step 8700, training accuracy 1
step 8800, training accuracy 1
step 8900, training accuracy 1
step 9000, training accuracy 1
step 9100, training accuracy 1
step 9200, training accuracy 1
step 9300, training accuracy 1
step 9400, training accuracy 1
step 9500, training accuracy 1
step 9600, training accuracy 1
step 9700, training accuracy 1
step 9800, training accuracy 1
step 9900, training accuracy 1
step 10000, training accuracy 1
step 10100, training accuracy 1
step 10200, training accuracy 1
step 10300, training accuracy 1
step 10400, training accuracy 1
step 10500, training accuracy 1
step 10600, training accuracy 0.96
step 10700, training accuracy 0.98
step 10800, training accuracy 1
step 10900, training accuracy 1
step 11000, training accuracy 1
step 11100, training accuracy 1
step 11200, training accuracy 1
step 11300, training accuracy 1
step 11400, training accuracy 1
step 11500, training accuracy 1
step 11600, training accuracy 1
step 11700, training accuracy 1
step 11800, training accuracy 1
step 11900, training accuracy 1
step 12000, training accuracy 0.98
step 12100, training accuracy 1
step 12200, training accuracy 1
step 12300, training accuracy 1
step 12400, training accuracy 1
step 12500, training accuracy 1
step 12600, training accuracy 0.96
step 12700, training accuracy 1
step 12800, training accuracy 1
step 12900, training accuracy 1
step 13000, training accuracy 0.98
step 13100, training accuracy 1
step 13200, training accuracy 1
step 13300, training accuracy 1
step 13400, training accuracy 1
step 13500, training accuracy 1
step 13600, training accuracy 0.98
step 13700, training accuracy 1
step 13800, training accuracy 0.98
step 13900, training accuracy 1
step 14000, training accuracy 1
step 14100, training accuracy 1
step 14200, training accuracy 1
step 14300, training accuracy 0.98
step 14400, training accuracy 1
step 14500, training accuracy 1
step 14600, training accuracy 1
step 14700, training accuracy 1
step 14800, training accuracy 1
step 14900, training accuracy 1
step 15000, training accuracy 0.98
step 15100, training accuracy 1
step 15200, training accuracy 1
step 15300, training accuracy 1
step 15400, training accuracy 1
step 15500, training accuracy 0.98
step 15600, training accuracy 1
step 15700, training accuracy 0.98
step 15800, training accuracy 1
step 15900, training accuracy 1
step 16000, training accuracy 1
step 16100, training accuracy 1
step 16200, training accuracy 1
step 16300, training accuracy 0.98
step 16400, training accuracy 1
step 16500, training accuracy 1
step 16600, training accuracy 1
step 16700, training accuracy 1
step 16800, training accuracy 1
step 16900, training accuracy 0.98
step 17000, training accuracy 1
step 17100, training accuracy 1
step 17200, training accuracy 1
step 17300, training accuracy 1
step 17400, training accuracy 1
step 17500, training accuracy 1
step 17600, training accuracy 1
step 17700, training accuracy 1
step 17800, training accuracy 1
step 17900, training accuracy 1
step 18000, training accuracy 1
step 18100, training accuracy 1
step 18200, training accuracy 0.98
step 18300, training accuracy 1
step 18400, training accuracy 1
step 18500, training accuracy 1
step 18600, training accuracy 1
step 18700, training accuracy 1
step 18800, training accuracy 1
step 18900, training accuracy 1
step 19000, training accuracy 1
step 19100, training accuracy 1
step 19200, training accuracy 1
step 19300, training accuracy 1
step 19400, training accuracy 1
step 19500, training accuracy 1
step 19600, training accuracy 1
step 19700, training accuracy 1
step 19800, training accuracy 1
step 19900, training accuracy 1
test accuracy 0.9924

Process finished with exit code 0

8-23 多通道MNIST

程序：

import tensorflow as tf
import numpy as np
# 导入 MINST 数据集
from tensorflow.examples.tutorials.mnist import input_data

mnist = input_data.read_data_sets("F:/shendu/MNIST_data/", one_hot=True)


def weight_variable(shape):
    initial = tf.truncated_normal(shape, stddev=0.1)
    return tf.Variable(initial)


def bias_variable(shape):
    initial = tf.constant(0.1, shape=shape)
    return tf.Variable(initial)


def conv2d(x, W):
    return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')


def max_pool_2x2(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                          strides=[1, 2, 2, 1], padding='SAME')


def avg_pool_7x7(x):
    return tf.nn.avg_pool(x, ksize=[1, 7, 7, 1],
                          strides=[1, 7, 7, 1], padding='SAME')


# tf Graph Input
x = tf.placeholder(tf.float32, [None, 784])  # mnist data维度 28*28=784
y = tf.placeholder(tf.float32, [None, 10])  # 0-9 数字=> 10 classes

W_conv1 = weight_variable([5, 5, 1, 32])
b_conv1 = bias_variable([32])

x_image = tf.reshape(x, [-1, 28, 28, 1])
h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
h_pool1 = max_pool_2x2(h_conv1)
######################################################多卷积核
W_conv2_5x5 = weight_variable([5, 5, 32, 32])
b_conv2_5x5 = bias_variable([32])
W_conv2_7x7 = weight_variable([7, 7, 32, 32])
b_conv2_7x7 = bias_variable([32])
h_conv2_5x5 = tf.nn.relu(conv2d(h_pool1, W_conv2_5x5) + b_conv2_5x5)
h_conv2_7x7 = tf.nn.relu(conv2d(h_pool1, W_conv2_7x7) + b_conv2_7x7)
h_conv2 = tf.concat([h_conv2_5x5, h_conv2_7x7], 3)

h_pool2 = max_pool_2x2(h_conv2)
#########################################################new 池化

W_conv3 = weight_variable([5, 5, 64, 10])
b_conv3 = bias_variable([10])
h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3) + b_conv3)

nt_hpool3 = avg_pool_7x7(h_conv3)  # 10
nt_hpool3_flat = tf.reshape(nt_hpool3, [-1, 10])
y_conv = tf.nn.softmax(nt_hpool3_flat)

keep_prob = tf.placeholder("float")

cross_entropy = -tf.reduce_sum(y * tf.log(y_conv))

# 不同的优化方法测测效果
# train_step = tf.train.GradientDescentOptimizer(1e-3).minimize(cross_entropy)
# train_step = tf.train.AdagradOptimizer(1e-5).minimize(cross_entropy)
train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)

correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))

# 启动session
with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for i in range(1000):  # 20000
        batch = mnist.train.next_batch(50)
        if i % 100 == 0:
            train_accuracy = accuracy.eval(feed_dict={
                x: batch[0], y: batch[1], keep_prob: 1.0})
            print("step %d, training accuracy %g" % (i, train_accuracy))
        train_step.run(feed_dict={x: batch[0], y: batch[1], keep_prob: 0.5})

    print("test accuracy %g" % accuracy.eval(feed_dict={
        x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}))

结果：

Extracting F:/shendu/MNIST_data/train-images-idx3-ubyte.gz

Extracting F:/shendu/MNIST_data/train-labels-idx1-ubyte.gz

Extracting F:/shendu/MNIST_data/t10k-images-idx3-ubyte.gz
Extracting F:/shendu/MNIST_data/t10k-labels-idx1-ubyte.gz

step 0, training accuracy 0.12
step 100, training accuracy 0.26
step 200, training accuracy 0.32
step 300, training accuracy 0.38
step 400, training accuracy 0.34
step 500, training accuracy 0.5
step 600, training accuracy 0.62
step 700, training accuracy 0.54
step 800, training accuracy 0.88
step 900, training accuracy 0.84
test accuracy 0.8438

Process finished with exit code 0
大春SSC
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
解析卷积神经网络——深度学习实践手册 .pdf
04-21
无水印版本
卷积神经网络——解决参数太多问题
新手的博客
05-10 1998
一、全连接网路的局限性 图像变大导致色彩书变多，不好解决 不便处理高维数据 对于比较复杂的高维数据，如果按照全连接的方法，则只能通过增加节点、增加层数的方式来解决。增加节点会引起参数过多的问题。由于隐藏层神经网络使用的是sigmod或tanh激活函数，其反向传播的有效成层数只能在4~6层左右。 二、理解卷积神经网络 三、网络结构 卷积神经网络的结构与全连接网络相比复杂很多。它的网络结构主要包括卷积层、池化层。细节又可以分为滤波器、步长、卷积操作、池化操作。 1.网络结构描述 对于一般的图片会使用多个卷积
8 卷积神经网络——解决参数太多问题
weixin_43318717的博客
06-26 1247
8-1 卷积函数使用 手动生成一个55的矩阵来模拟图片，定义一个22的卷积核，来测试tf.nn.conv2d函数里的不同参数，验证其输出结果。 程序： import tensorflow as tf #1 定义输入变量 [batch, in_height, in_width, in_channels] [训练时一个batch的图片数量, 图片高度, 图片宽度, 图像通道数] input = tf....
全连接神经网络(DNN)
凌逆战的博客
11-18 7503
全连接神经网络(DNN)是最朴素的神经网络，它的网络参数最多，计算量最大。 网络结构 　　DNN的结构不固定，一般神经网络包括输入层、隐藏层和输出层，一个DNN结构只有一个输入层，一个输出层，输入层和输出层之间的都是隐藏层。每一层神经网络有若干神经元(下图中蓝色圆圈)，层与层之间神经元相互连接，层内神经元互不连接，而且下一层神经元连接上一层所有的神经元。 　　隐藏层比较多(>...
经网络中隐层数和隐层节点数问题的讨论
lujiandong1的专栏
04-07 6652
神经网络中隐层数和隐层节点数问题的讨论 一 隐层数         一般认为，增加隐层数可以降低网络误差（也有文献认为不一定能有效降低），提高精度，但也使网络复杂化，从而增加了网络的训练时间和出现“过拟合”的倾向。一般来讲应设计神经网络应优先考虑3层网络（即有1个隐层）。一般地，靠增加隐层节点数来获得较低的误差，其训练效果要比增加隐层数更容易实现。对于没有隐层的神经网络模型，实际上就是一个线性
关于深度学习神经网络模型训练，参数过大，导致显卡内存溢出问题的总结
qq_41166909的博客
10-17 1万+
实验是利用LeNet-5模型,计算手写数字集得误差。结果刚加载模型显卡内存就溢出。 报错如下： 先用keras来计算网络的大小，参数的个数。 (50000, 32, 32, 3) (50000,) (10000, 32, 32, 3) (10000,) sample: (512, 32, 32, 3) (512,) tf.Tensor(-1.0, shape=(), dtype=float32) tf.Tensor(1.0, shape=(), dtype=float32) Model: "re
深度学习——卷积神经网络
01-20
文章目录卷积神经网络一、卷积神经网络基础1.基础知识2.卷积层的简洁实现3.池化二、LeNet三、常见的一些卷积神经网络1.AlexNet2.VGG3.NiN4.GoogLeNet   一、卷积神经网络基础 1.基础知识 二维互相关(cross-...
PyTorch卷积神经网络案例分析——LeNet
01-06
#LeNet 是整个卷积神经网络的开山之作，一共有七层，其实2层卷积和2层池化层交替出现， #最后输出三层全连接层得到整体的结果 import numpy as np import torch from torch.autograd import Variable from torch ...
TensorFlow1.x入门（9）——卷积神经网络（CNN）
12-21
卷积神经网络（Convolutional Neural Networks, CNN）是深度学习领域内的一个重要的组成构建。CNN现在大量的用于图像识别领域，但是在自然语言处理和语音识别领域作为特征抽取器有广泛的应用。 一般情况下，提到卷积...
AI学习笔记——卷积神经网络（CNN）
02-24
本文来自简书，本文主要介绍了一个完整的卷积神经网络(CNN)，卷积的滤波器（Filter）是如何工作的呢？请参见下文。上篇文章简单地地介绍了神经网络和深度学习，在神经网络中，每一层的每个神经元都与下一层的每个...
sudoku, 神经网络能破解数独？.zip
09-17
sudoku, 神经网络能破解数独？ 可以卷积神经网络裂纹数独拼图？数独是一个常见的数字谜题，要求你用数字填充一个X9网格，使每个列，每一行，每一行的数字都包含所有数字，从 1到 9. 有多种方法可以解决这些问题，包括计算方法。 在这个项目中，简单的卷积神经网络具有无规则后处
卷积神经网络——解决参数太多的问题
qq_43161793的博客
11-05 1705
卷积函数 tf.nn.conv2d(input,filter,strides,padding,use_cudnn_on_gpu=None,name=None) #input:[batch,in_height,in_width,in_channels] #filter:[filter_height,filter_width,in_channels,out_channels]
十、如何选择神经网络的超参数
热门推荐
dugudaibo的博客
08-18 5万+
本节主要介绍了如何选择神经网络中的超参数。根据神经网络中超参数的特性对超参数进行分类，并给出了大致三种调整超参数的方法。首先根据机理确定激活函数的种类，代价函数的种类，权重初始化的方法，输出层的编码方式；其次根据宽泛策略给出一个结构较为简单神经网络，在这里可以确定网络层数，神经元个数；最后依次确定学习率，minibatch，lambda和回合数。
深度学习应用9.1从全连接到卷积神经网络-解决参数过多
roymustang的博客
03-13 364
2024-2030全球与中国盐氯化系统市场现状及未来发展趋势.docx
04-24
2024-2030全球与中国盐氯化系统市场现状及未来发展趋势
基于深度学习的积灰检测识别-图像分类源码+数据集.zip
最新发布
04-24
详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；详情请查看资源内容中使用说明；
沈阳药科大学-答辩通用PPT模板我给母校送模板作品.pptx
04-24
PPT模板，答辩PPT模板，毕业答辩，学术汇报，母校模板，我给母校送模板作品，周会汇报，开题答辩，教育主题模板下载。PPT素材下载。
微信备忘录小程序源码 作业设计demo 计算机专业作业
04-24
微信备忘录小程序源码 作业设计demo 微信备忘录小程序是一种便捷的个人记事应用，它允许用户在微信内快速记录和查看备忘录。以下是对微信备忘录小程序的简要介绍： --- **微信备忘录小程序** 微信备忘录小程序为用户提供了一个简单、直观的记事平台。用户可以通过这个小程序记录日常事务、重要提醒和个人笔记，非常适合忙碌的现代生活节奏。 主要特点包括： 1. **快速记录**：用户可以迅速添加文本、列表或语音备忘录。 2. **定时提醒**：为每个备忘录设置提醒时间，确保不会错过任何重要事项。 3. **个性化分类**：支持自定义分类，便于管理和查找备忘录。 4. **界面友好**：清晰的界面设计，操作简便，无需复杂的学习过程。 5. **数据同步**：通过微信账号登录，实现备忘录的云端同步，方便在不同设备间切换使用。 此外，小程序还具备以下优势： - **隐私保护**：备忘录内容仅对用户本人可见，保障个人隐私。 - **无广告干扰**：提供一个无广告的清爽记事环境。 - **离线存储**：即使在无网络环境下，也能正常使用，记录的内容会在联网后自动同步。 微信备忘录小程序是日
卷积神经网络——全连接操作
09-11
全连接操作是卷积神经网络（Convolutional Neural Network, CNN）中的一种常见操作，也被称为全连接层或者密集连接层。在卷积神经网络中，全连接层通常紧跟在卷积层和池化层之后。 在卷积神经网络中，卷积层和池化层主要用于提取图像的局部特征，而全连接层则用于将这些局部特征进行组合和整合，最终输出分类结果或者预测值。 全连接操作的过程是将前一层的所有节点与当前层的所有节点相连接。具体而言，对于前一层的每个节点，都与当前层的每个节点相连，并且每条连接都有一个权重。这样，当前层的每个节点都能够接收来自前一层所有节点的信息。 全连接操作可以将卷积神经网络中提取到的局部特征进行整合和组合，从而得到更高级别的语义信息。通过训练过程中不断调整权重，全连接层可以学习到图像中不同特征之间的关系，并根据这些关系做出分类或者预测。 需要注意的是，全连接操作可能会导致模型参数过多，增加计算量和内存消耗。因此，在设计卷积神经网络时，需要根据具体任务和数据集的特点，合理设置全连接层的大小和数量，以达到较好的性能和效果。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值