Tensorflow入门五-卷积神经网络(断点续训实现,基本概念、tf卷积函数、cifar数据集百度云资源及tf实现)

上一篇:Tensorflow入门四-手写数据集MNIST识别神经网络并实现存取模型(one-hot编码,softmax,交叉熵)https://blog.csdn.net/qq_36187544/article/details/89683997

下一篇:Tensorflow入门六-图像处理及Python实现(DeepDream,1×1卷积核,捷径连接)https://blog.csdn.net/qq_36187544/article/details/89742830

由于cifar数据集通过python下载实在太慢了,给一个百度云链接链接:链接:https://pan.baidu.com/s/1y0nVg7stYz4OPwxc4bh8pg 
提取码:cvee 

断点续训很重要,可多次非连续训练,适合实际工作


目录

卷积神经网络的基本概念

tensorflow卷积函数

CIFAR-10数据集

TensorFlow实现CNN并实现断点续训


卷积神经网络的基本概念

卷积神经网络(CNN),计算机视觉入门,卷积的理解:机器学习(14)--经典边缘检测canny算法(计算机视觉从0到1)https://blog.csdn.net/qq_36187544/article/details/89548363

相较于传统的全连接神经网络(DNN)而言,卷积神经网络利用局部连接参数共享的方式降低了参数的数量,实现速度更快,网络可以更深更广!

padding(填充):图像在进行卷积后往往图像大小会改变,采用填充的方式使图像大小不变:

多通道卷积:比如常见的RGB三通道,每个通道采用一个卷积核生成对应图样,再进行合成。可以保留更多的特征(比如一个保留纵向特征,一个保留横向特征等),当然可以加偏置等操作

池化降采样):目的在于减小特征数量,防止过拟合。常用方法--均值池化(对区域内像素点取均值,特征数据对背景敏感),最大池化(取最大值,得到特征对纹理特征信息更加敏感)

步长(stribe):表示卷积核在图片移动的格数,通过不同步长可得到不同尺寸卷积输出结果,步长大于1可降维,(池化也是降维)

总之,卷积神经网络中,卷积层为了增强特征,降噪,池化层为了防止过拟合,减少特征数量


tensorflow卷积函数

需要注意的是Input一定是一个张量,进入图像需要进行图像处理


CIFAR-10数据集

一个适用于识别普适物体的小型数据集,10个类别的RGB彩图,尺寸32×32,训练图片50000张,测试10000张

可查看cifar10数据集https://www.cs.toronto.edu/~kriz/cifar.html

由于这个数据集通过程序下载很慢,文章开头有百度云链接可下载放在相关目录下。


TensorFlow实现CNN并实现断点续训

什么叫断点续训?程序可以保存模型,为了防止意外情况发生,需要不时保存模型,并在下一次训练时调用已保存的数据继续进行训练,可以防止各种意外!

实现的CNN结构如下:

'''
cifar数据集卷积神经网络,断点续训

'''
import matplotlib.pyplot as plt
import tensorflow as tf
import numpy as np
from time import time
import os
import urllib.request
import tarfile
import pickle as p
from sklearn.preprocessing import OneHotEncoder

#下载,如果是第一次执行下载要很久!注意同级目录下一定要有data文件夹,否则会提示文件或文件夹不存在
from tensorflow.contrib.timeseries import predict_continuation_input_fn

url = 'https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz'
filepath = 'data/cifar-10-python.tar.gz'
if not os.path.isfile(filepath):
    result = urllib.request.urlretrieve(url,filepath)
    print("download:",result)
else:
    print("file existed")

#解压
if not os.path.exists("data/cifar-10-batches-py"):
    tfile = tarfile.open("data/cifar-10-python.tar.gz",'r:gz')
    result = tfile.extractall("data/")
    print("Extracted to ./data/cifar-10-batches-py")
else:
    print("directory existed")

#读取单批次数据样本,一个样本由标签和图像数据组成
def load_CIFAR_batch(filename):
    with open(filename,'rb') as f:
        data_dict = p.load(f,encoding='bytes')
        images = data_dict[b'data']
        labels = data_dict[b'labels']

        #把原始数据结构调整为:BCWH
        images = images.reshape(10000,3,32,32)
        #tensorflow处理图像数据结构为:BWHC
        images = images.transpose(0,2,3,1)
        labels = np.array(labels)
        return images,labels

#读取完整数据集
def load_CIFAR_data(data_dir):
    images_train=[]
    labels_train=[]
    for i in range(5):
        f = os.path.join(data_dir, "data_batch_%d"%(i+1))
        print("loading",f)
        image_batch,label_batch = load_CIFAR_batch(f)
        images_train.append(image_batch)
        labels_train.append(label_batch)
        Xtrain = np.concatenate(images_train)
        Ytrain = np.concatenate(labels_train)
        del image_batch,label_batch
    Xtest,Ytest = load_CIFAR_batch(os.path.join(data_dir,'test_batch'))
    print("finished loadding CIFAR-10 data")
    return Xtrain,Ytrain,Xtest,Ytest

data_dir = 'data/cifar-10-batches-py/'
Xtrain,Ytrain,Xtest,Ytest = load_CIFAR_data(data_dir)

#查看单项image和label
# plt.imshow(Xtrain[5])
# plt.show()

#定义标签早点
label_dict = {0:'airplane',1:'automobile',2:'bird',3:'cat',4:'deer',5:'dog',6:'frog',7:'horse',8:'ship',9:'truck'}
#定义显示图像及标签数据
def plot_images_labels_prediction(images,labels,prediction,idx,num=10):
    fig = plt.gcf()
    fig.set_size_inches(12,6)
    if num>10:
        num = 10
    for i in range(num):
        ax = plt.subplot(2,5,1+i)
        ax.imshow(images[idx],cmap = 'binary')
        title = str(i)+','+label_dict[labels[idx]]
        if len(prediction)>0:
            title += '=>'+label_dict[prediction[idx]]
        ax.set_title(title,fontsize=10)
        idx+=1
    plt.show()

#显示图像
#plot_images_labels_prediction(Xtest,Ytest,[],0,10)

#图像数据预处理
#数字标准化
Xtrain_normalize = Xtrain.astype('float32')/255.0
Xtest_normalize = Xtest.astype('float32')/255.0
#标签转独热编码
encoder = OneHotEncoder(sparse=False)
yy = [[0],[1],[2],[3],[4],[5],[6],[7],[8],[9]]
encoder.fit(yy)
Ytrain_reshape = Ytrain.reshape(-1,1)
Ytrain_onehot = encoder.transform(Ytrain_reshape)
Ytest_reshape = Ytest.reshape(-1,1)
Ytest_onehot = encoder.transform(Ytest_reshape)


tf.reset_default_graph()
#定义权值,偏置,卷积,池化
def weight(shape):
    return tf.Variable(tf.truncated_normal(shape,stddev=0.1),name='W')
def bias(shape):
    return tf.Variable(tf.constant(0.1,shape=shape),name='b')
def conv2d(x,W):
    return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME')
def max_pool_2(x):
    return tf.nn.max_pool(x,ksize=[1,2,2,1],strides=[1,2,2,1],padding='SAME')

#定义网络结构
with tf.name_scope('input_layer'):
    x = tf.placeholder('float',shape=[None,32,32,3],name="x")
with tf.name_scope('conv_1'):
    W1 = weight([3,3,3,32]) #[k_width,k_height,input_chn,output_chn]
    b1 = bias([32])
    conv_1 = conv2d(x,W1)+b1
    conv_1 = tf.nn.relu(conv_1)
with tf.name_scope('pool_1'):
    pool_1 = max_pool_2(conv_1)
with tf.name_scope('conv_2'):
    W2 = weight([3,3,32,64])
    b2 = bias([64])
    conv_2 = conv2d(pool_1,W2)+b2
    conv_2 = tf.nn.relu(conv_2)
with tf.name_scope('pool_2'):
    pool_2 = max_pool_2(conv_2)
with tf.name_scope('fc'):
    W3 = weight([4096,128])
    b3 = bias([128])
    flat = tf.reshape(pool_2,[-1,4096])
    h = tf.nn.relu(tf.matmul(flat,W3)+b3)
    h_dropout = tf.nn.dropout(h,keep_prob=0.8)
with tf.name_scope('output_layer'):
    W4 = weight([128,10])
    b4 = bias([10])
    pred = tf.nn.softmax(tf.matmul(h_dropout,W4)+b4)

# 构建模型
with tf.name_scope('optimizer'):
    y = tf.placeholder('float',shape=[None,10],name='label')
    loss_function = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred,labels=y))
    optimizer = tf.train.AdamOptimizer(learning_rate=0.0001).minimize(loss_function)

#定义准确率
with tf.name_scope('evalution'):
    correct_prediction = tf.equal(tf.argmax(pred,1),tf.argmax(y,1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction,'float'))

train_epochs = 8
batch_size = 50
total_batch = int(len(Xtrain)/batch_size)#一轮训练多少批次
epoch_list,accuracy_list,loss_list = [],[],[]

epoch = tf.Variable(0,name='epoch',trainable=False)

startTime = time()
sess = tf.Session()
init = tf.global_variables_initializer()
sess.run(init)

#断点续训
ckpt_dir = 'CIFAR-10_log/'
if not os.path.exists(ckpt_dir):
    os.makedirs(ckpt_dir)
#生成saver
saver = tf.train.Saver(max_to_keep=1)
ckpt = tf.train.latest_checkpoint(ckpt_dir)
if ckpt != None:
    saver.restore(sess,ckpt)
else:
    print("training from scratch")

#获取续训参数
start = sess.run(epoch)
print("Training start from {} epoch".format(start+1))

#迭代训练
def get_train_batch(number,batch_size):
    return Xtrain_normalize[number*batch_size:(number+1)*batch_size],Ytrain_onehot[number*batch_size:(number+1)*batch_size]

for ep in range(start,train_epochs):
    for i in range(total_batch):
        batch_x,batch_y = get_train_batch(i,batch_size)
        sess.run(optimizer,feed_dict={x:batch_x,y:batch_y})
        if i % 100 == 0:
            print("step{}".format(i),"finished")
    loss,acc = sess.run([loss_function,accuracy],feed_dict={x:batch_x,y:batch_y})
    epoch_list.append(ep+1)
    loss_list.append(loss)
    accuracy_list.append(acc)
    print("Train epoch:","%02d"%(sess.run(epoch)+1),"Loss= ","{:.6f}".format(loss),"Accuracy=",acc)
    #保存检查点
    saver.save(sess,ckpt_dir+"CIFAR10_cnn_model.cpkt",global_step=ep+1)
    sess.run(epoch.assign(ep+1))
duration = time() - startTime
print("training takes time:",duration)

#可视化损失值
fig = plt.gcf()
fig.set_size_inches(4,2)
plt.plot(epoch_list,loss_list,label='loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['loss'],loc='upper right')
plt.show()

#可视化准确率
plt.plot(epoch_list,accuracy_list,label="accuracy")
fig = plt.gcf()
fig.set_size_inches(4,2)
plt.ylim(0.1,1)
plt.ylabel("accuracy")
plt.xlabel('epoch')
plt.legend()
plt.show()

#评估模型和预测
test_total_batch = int(len(Xtest_normalize)/batch_size)
test_acc_sum = 0.0
for i in range(test_total_batch):
    test_image_batch = Xtest_normalize[i*batch_size:(i+1)*batch_size]
    test_label_batch = Ytest_onehot[i*batch_size:(i+1)*batch_size]
    test_batch_acc = sess.run(accuracy, feed_dict={x:test_image_batch,y:test_label_batch})
    test_acc_sum += test_batch_acc
test_acc = float(test_acc_sum/test_total_batch)
print("test accuracy :{:.6f}".format(test_acc))

#利用模型进行预测
test_pred = sess.run(pred,feed_dict={x:Xtest_normalize[:10]})
prediction_result = sess.run(tf.argmax(test_pred,1))
plot_images_labels_prediction(Xtest,Ytest,prediction_result,0,10)


控制台不断输出:

我们可以随时将程序停掉,并查看目录下日志保存情况,直接再次执行即可断点续训*(可以发现初始从第二代开始):

可视化展示结果,电脑垃圾,运行太慢,执行了4代差不多了,截图over,如果迭代次数够久,可以看见收敛性等:

多数预测正确,部分预测错误:

  • 2
    点赞
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值