TensorFlow学习－－卷积神经网络训练CIFAR-10数据集

最新推荐文章于 2024-09-17 21:17:40 发布

阿卡蒂奥

最新推荐文章于 2024-09-17 21:17:40 发布

阅读量4.9k

点赞数 1

分类专栏： TensorFlow 文章标签： tensorflow cifar-10 卷积神经网络

本文链接：https://blog.csdn.net/akadiao/article/details/78505441

版权

TensorFlow 专栏收录该内容

34 篇文章 9 订阅

订阅专栏

CIFAR-10数据集

CIFAR-10数据集包含10个类的60000张32x32的彩色图像，每个类有6000张图像。有50000张训练图像和10000张测试图像。
10个分类明细及对应的部分图片：

这里写图片描述

CIFAR-10数据集连接

卷积神经网络训练CIFAR-10数据集

论文ImageNet Classification with Deep Convolutional
Neural Networks
LRN（Local Response Normalization）最早在Alex的这篇论文中提出

这里写图片描述

$\ a_{x,y}^{i}$ 表示(x,y)处以 $\ kernel_{i}$ 计算后经过激励的输出;
$\ b_{x,y}^{i}$ 表示经过LRN层后作为下一层输入的数据；
$\ N$ 表示该层的特征图(feature map)总数
$\ n$ 表示取特征图中间左右各 $\ n/2$ 个特征图求均值．

LNR的作用是增强响应比较大的神经元，并抑制增强响应比较小的神经元，增强泛化能力．LRN 会从附近的多个卷积核的响应中挑选比较大的反馈；因此，适用于对没有上限边界的激活函数（如Relu ），但不适合有固定边界且能抑制过大反馈的激活函数（如sigmoid）。

代码及注释

下载TensorFlow Models库，获得提供DIFAR-10数据的类：
tensorflow/models链接

# CIFAR-10数据集训练
import sys
sys.path.append("/home/w/mycode/TensorFlow/CIFAR10/models/tutorials/image/cifar10")
# 下载和读取CIFAR-10的类
import cifar10,cifar10_input
import tensorflow as tf
import numpy as np
import time

# 训练轮数
max_steps = 3000
batch_size = 128
# 下载DIFAR-10数据的默认路径
data_dir = '/tmp/cifar10_data/cifar-10-batches-bin'

# L1正则会制造稀疏的特征，大部分无用特征的权重会被置0
# L2正则会让特征不过大，使得特征的权重比较均匀

# 使用正太分布初始化权重并添加L2正则化,使用w1控制L2损失的大小
def variable_with_weight_loss(shape, stddev, w1):
    # 从截断的(2个标准差以内)正态分布中输出随机值
    var = tf.Variable(tf.truncated_normal(shape, stddev=stddev))
    if w1 is not None:
        # l2_loss(var)*w1
        weight_loss = tf.multiply(tf.nn.l2_loss(var), w1, name='weight_loss')
        # 使用默认图
        tf.add_to_collection('losses', weight_loss)
    return var

# 从Alex的网站下载并解压到默认位置
cifar10.maybe_download_and_extract()
# 使用Reader操作构造CIFAR训练需要的数据(特征及其对应的label)
# 并对数据进行了数据增强(水平翻转/随机对比度亮度/随机裁剪)以及数据的标准化
images_train, labels_train = cifar10_input.distorted_inputs(data_dir=data_dir,batch_size=batch_size)
# 使用Reader操作构建CIFAR评估的输入(裁剪图像中间24*24大小的块并进行数据标准化)
images_test, labels_test =cifar10_input.inputs(eval_data=True, data_dir=data_dir, batch_size=batch_size)

# 输入图像占位符(24*24 3通道)
image_holder = tf.placeholder(tf.float32, [batch_size, 24, 24, 3])
# 输入标签占位符
label_holder = tf.placeholder(tf.int32, [batch_size])

# 卷积层1
# 64个5*5的卷积核3通道,不对第一个卷积层的权重加L2正则
weight1 = variable_with_weight_loss(shape=[5, 5, 3, 64], stddev=5e-2, w1=0.0)
# 卷积步长为1模式为SAME
kernel1 = tf.nn.conv2d(image_holder, weight1, [1, 1, 1, 1], padding='SAME')
# bias为0
bias1 = tf.Variable(tf.constant(0.0, shape=[64]))
# Adds bias to value
conv1 = tf.nn.relu(tf.nn.bias_add(kernel1, bias1))
# 最大池化　大小3*3步长2*2
pool1 = tf.nn.max_pool(conv1, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1], padding='SAME')
# 使用LRN对结果进行处理-Local Response Normalization-本地响应标准化
# 增强大的抑制小的,增强泛化能力
norm1 = tf.nn.lrn(pool1, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)

# 卷积层2
# 64个5*5的卷积核64通道,不加L2正则
weight2 = variable_with_weight_loss(shape=[5, 5, 64, 64], stddev=5e-2, w1=0.0)
# 卷积步长为1模式为SAME
kernel2 = tf.nn.conv2d(norm1, weight2, [1, 1, 1, 1], padding='SAME')
# bias为0.1
bias2 = tf.Variable(tf.constant(0.1, shape=[64]))
# Adds bias to value
conv2 = tf.nn.relu(tf.nn.bias_add(kernel2, bias2))
# LRN-本地响应标准化
norm2 = tf.nn.lrn(conv2, 4, bias=1.0, alpha=0.001/0.9, beta=0.75)
# 最大池化　大小3*3步长2*2
pool2 = tf.nn.max_pool(norm2, ksize=[1, 3, 3, 1], strides=[1, 2, 2, 1], padding='SAME')

# 全连接层
# 将样本变成一维向量
reshape = tf.reshape(pool2, [batch_size, -1])
# 数据扁平化后的长度
dim = reshape.get_shape()[1].value
# weight初始化
weight3 = variable_with_weight_loss(shape=[dim, 384], stddev=0.04, w1=0.004)
# bias初始化
bias3 = tf.Variable(tf.constant(0.1, shape=[384]))
local3 = tf.nn.relu(tf.matmul(reshape, weight3) + bias3)

# 隐含节点数降为192
weight4 = variable_with_weight_loss(shape=[384, 192], stddev=0.04, w1=0.004)
bias4 = tf.Variable(tf.constant(0.1, shape=[192]))
local4 = tf.nn.relu(tf.matmul(local3, weight4) + bias4)

# 最终输出10分类,正太分布标准差设为上一隐含层节点数的倒数,不计入L2正则
weight5 = variable_with_weight_loss(shape=[192, 10], stddev=1/192.0, w1=0.0)
bias5 = tf.Variable(tf.constant(0.0, shape=[10]))
logits = tf.add(tf.matmul(local4, weight5), bias5)


# 计算CNN的loss
def loss(logits, labels):
    labels = tf.cast(labels, tf.int64)
    # 计算logits和labels之间的稀疏softmax交叉熵
    cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(
        logits=logits, labels=labels, name='cross_entropy_per_example')
    # 计算cross_entropy均值
    cross_entropy_mean = tf.reduce_mean(cross_entropy, name='cross_entropy')
    # 将cross_entropy的loss添加到整体的loss里
    tf.add_to_collection('losses', cross_entropy_mean)
    # 将整体losses的collection中的全部loss求和
    return tf.add_n(tf.get_collection('losses'), name='total_loss')


# 将logits节点label_holder和传入loss函数获得最终的loss
loss = loss(logits, label_holder)
# 优化器选择Adam,学习率选1e-3
train_op = tf.train.AdamOptimizer(1e-3).minimize(loss)
# 求top k的准确率(默认top1即输出分数最高的一类)
top_k_op = tf.nn.in_top_k(logits, label_holder, 1)


# 创建默认的session()
sess = tf.InteractiveSession()
# 初始化全部的模型参数
tf.global_variables_initializer().run()

# 启动图片数据增强的线程队列
tf.train.start_queue_runners()

for step in range(max_steps):
    start_time = time.time()
    # 使用sess的run方法执行images_train和labels_train的计算
    image_batch, label_batch = sess.run([images_train, labels_train])
    #
    _, loss_value = sess.run([train_op, loss],
              feed_dict={image_holder: image_batch, label_holder:label_batch})
    # 记录每个step的时间
    duration = time.time() - start_time
    if step % 10 == 0:
        # 每秒训练的样本数量
        examples_per_sec = batch_size / duration
        # 训练每个batch的时间
        sec_per_batch = float(duration)
        format_str = ('step %d, lass=%.2f (%.1f examples/sec; %.3f sec/batch)')
        print format_str % (step, loss_value, examples_per_sec, sec_per_batch)


# 测试集样本数量
num_examples = 10000
import math
# 总共多少个batch
num_inter = int(math.ceil(num_examples / batch_size))
true_count = 0
total_sample_count = num_inter * batch_size
step = 0
while step < num_inter:
    # 使用sess的run方法获取images_test和labels_test的batch
    image_batch, label_batch = sess.run([images_test, labels_test])
    # 预测正确的样本数量
    predictions = sess.run([top_k_op], feed_dict={image_holder: image_batch,
                            label_holder: label_batch})
    # 汇总预测正确的结果
    true_count += np.sum(predictions)
    step += 1

# 准确率评测结果
prediction = true_count / total_sample_count
print 'precision @ 1 = %.3f' % prediction

使用的技巧有：

对weights进行了L2正则化．
对图片进行了翻转　随机剪切等数据增强，制造了更多样本．
每个卷积-最大池化层后面使用了LRN层,增强了模型的泛化能力．

下载解压后得到的数据：
这里写图片描述

这里写图片描述

输出结果：

Downloading cifar-10-binary.tar.gz 100.0%
Successfully downloaded cifar-10-binary.tar.gz 170052171 bytes.
Filling queue with 20000 CIFAR images before starting to train. This will take a few minutes.

step 0, lass=4.67 (25.7 examples/sec; 4.989 sec/batch)
step 10, lass=3.69 (141.3 examples/sec; 0.906 sec/batch)
step 20, lass=3.20 (152.7 examples/sec; 0.838 sec/batch)
step 30, lass=2.72 (116.9 examples/sec; 1.095 sec/batch)
step 40, lass=2.48 (113.8 examples/sec; 1.124 sec/batch)
step 50, lass=2.38 (145.6 examples/sec; 0.879 sec/batch)
step 60, lass=2.40 (106.2 examples/sec; 1.205 sec/batch)
step 70, lass=2.10 (142.8 examples/sec; 0.896 sec/batch)
step 80, lass=2.01 (148.4 examples/sec; 0.862 sec/batch)
step 90, lass=2.04 (89.5 examples/sec; 1.430 sec/batch)
step 100, lass=1.97 (127.1 examples/sec; 1.007 sec/batch)
step 110, lass=1.92 (114.9 examples/sec; 1.114 sec/batch)
step 120, lass=1.93 (104.7 examples/sec; 1.223 sec/batch)
step 130, lass=1.85 (106.6 examples/sec; 1.201 sec/batch)
step 140, lass=1.82 (98.9 examples/sec; 1.294 sec/batch)
step 150, lass=1.92 (105.4 examples/sec; 1.214 sec/batch)
step 160, lass=1.89 (118.9 examples/sec; 1.077 sec/batch)
step 170, lass=1.68 (115.2 examples/sec; 1.111 sec/batch)
step 180, lass=1.76 (132.7 examples/sec; 0.965 sec/batch)
step 190, lass=1.85 (85.1 examples/sec; 1.504 sec/batch)
step 200, lass=1.75 (98.5 examples/sec; 1.299 sec/batch)
step 210, lass=1.81 (109.9 examples/sec; 1.164 sec/batch)
step 220, lass=1.75 (98.6 examples/sec; 1.298 sec/batch)
step 230, lass=1.79 (99.8 examples/sec; 1.282 sec/batch)
step 240, lass=1.64 (114.7 examples/sec; 1.116 sec/batch)
step 250, lass=1.91 (116.4 examples/sec; 1.100 sec/batch)
step 260, lass=1.68 (104.0 examples/sec; 1.231 sec/batch)
step 270, lass=1.45 (101.3 examples/sec; 1.264 sec/batch)
step 280, lass=1.45 (100.2 examples/sec; 1.277 sec/batch)
step 290, lass=1.67 (70.3 examples/sec; 1.820 sec/batch)
step 300, lass=1.73 (100.5 examples/sec; 1.273 sec/batch)
step 310, lass=1.61 (97.2 examples/sec; 1.317 sec/batch)
step 320, lass=1.63 (93.4 examples/sec; 1.371 sec/batch)
step 330, lass=1.56 (79.9 examples/sec; 1.601 sec/batch)
step 340, lass=1.45 (78.9 examples/sec; 1.622 sec/batch)
step 350, lass=1.54 (85.7 examples/sec; 1.494 sec/batch)
step 360, lass=1.43 (94.6 examples/sec; 1.354 sec/batch)
step 370, lass=1.54 (134.3 examples/sec; 0.953 sec/batch)
step 380, lass=1.52 (70.5 examples/sec; 1.816 sec/batch)
step 390, lass=1.70 (71.0 examples/sec; 1.803 sec/batch)
step 400, lass=1.63 (67.8 examples/sec; 1.887 sec/batch)
step 410, lass=1.35 (72.7 examples/sec; 1.760 sec/batch)
step 420, lass=1.54 (71.4 examples/sec; 1.794 sec/batch)
step 430, lass=1.60 (61.4 examples/sec; 2.084 sec/batch)
step 440, lass=1.65 (69.4 examples/sec; 1.844 sec/batch)
step 450, lass=1.38 (70.7 examples/sec; 1.810 sec/batch)
step 460, lass=1.31 (67.5 examples/sec; 1.898 sec/batch)
step 470, lass=1.55 (53.7 examples/sec; 2.385 sec/batch)
step 480, lass=1.42 (55.3 examples/sec; 2.314 sec/batch)
step 490, lass=1.33 (52.6 examples/sec; 2.432 sec/batch)
step 500, lass=1.31 (61.5 examples/sec; 2.082 sec/batch)
step 510, lass=1.68 (69.3 examples/sec; 1.846 sec/batch)
step 520, lass=1.47 (57.8 examples/sec; 2.213 sec/batch)
step 530, lass=1.50 (60.7 examples/sec; 2.109 sec/batch)
step 540, lass=1.25 (57.3 examples/sec; 2.236 sec/batch)
step 550, lass=1.61 (51.4 examples/sec; 2.492 sec/batch)
step 560, lass=1.33 (53.3 examples/sec; 2.403 sec/batch)
step 570, lass=1.48 (53.9 examples/sec; 2.373 sec/batch)
step 580, lass=1.42 (51.9 examples/sec; 2.468 sec/batch)
step 590, lass=1.49 (61.7 examples/sec; 2.073 sec/batch)
step 600, lass=1.52 (50.6 examples/sec; 2.532 sec/batch)
step 610, lass=1.31 (64.2 examples/sec; 1.994 sec/batch)
step 620, lass=1.61 (70.4 examples/sec; 1.819 sec/batch)
step 630, lass=1.30 (69.8 examples/sec; 1.835 sec/batch)
step 640, lass=1.45 (71.3 examples/sec; 1.796 sec/batch)
step 650, lass=1.43 (71.9 examples/sec; 1.780 sec/batch)
step 660, lass=1.58 (74.2 examples/sec; 1.726 sec/batch)
step 670, lass=1.29 (71.5 examples/sec; 1.790 sec/batch)
step 680, lass=1.11 (72.0 examples/sec; 1.777 sec/batch)
step 690, lass=1.20 (61.7 examples/sec; 2.074 sec/batch)
step 700, lass=1.36 (70.1 examples/sec; 1.827 sec/batch)
step 710, lass=1.39 (66.9 examples/sec; 1.914 sec/batch)
step 720, lass=1.49 (69.8 examples/sec; 1.833 sec/batch)
step 730, lass=1.63 (67.3 examples/sec; 1.901 sec/batch)
step 740, lass=1.42 (69.1 examples/sec; 1.852 sec/batch)
step 750, lass=1.34 (68.4 examples/sec; 1.871 sec/batch)
step 760, lass=1.33 (69.4 examples/sec; 1.843 sec/batch)
step 770, lass=1.48 (70.0 examples/sec; 1.830 sec/batch)
step 780, lass=1.34 (70.8 examples/sec; 1.808 sec/batch)
step 790, lass=1.39 (72.0 examples/sec; 1.778 sec/batch)
step 800, lass=1.31 (71.6 examples/sec; 1.788 sec/batch)
step 810, lass=1.30 (69.8 examples/sec; 1.834 sec/batch)
step 820, lass=1.27 (72.3 examples/sec; 1.770 sec/batch)
step 830, lass=1.36 (70.0 examples/sec; 1.830 sec/batch)
step 840, lass=1.36 (54.0 examples/sec; 2.369 sec/batch)
step 850, lass=1.40 (59.3 examples/sec; 2.157 sec/batch)
step 860, lass=1.18 (66.9 examples/sec; 1.912 sec/batch)
step 870, lass=1.29 (52.6 examples/sec; 2.433 sec/batch)
step 880, lass=1.18 (71.2 examples/sec; 1.799 sec/batch)
step 890, lass=1.28 (67.8 examples/sec; 1.888 sec/batch)
step 900, lass=1.13 (72.2 examples/sec; 1.773 sec/batch)
step 910, lass=1.26 (67.1 examples/sec; 1.908 sec/batch)
step 920, lass=1.30 (69.5 examples/sec; 1.841 sec/batch)
step 930, lass=1.23 (71.3 examples/sec; 1.795 sec/batch)
step 940, lass=1.20 (71.8 examples/sec; 1.782 sec/batch)
step 950, lass=1.50 (71.0 examples/sec; 1.803 sec/batch)
step 960, lass=1.31 (74.3 examples/sec; 1.724 sec/batch)
step 970, lass=1.12 (75.0 examples/sec; 1.707 sec/batch)
step 980, lass=1.31 (72.3 examples/sec; 1.770 sec/batch)
step 990, lass=1.25 (73.3 examples/sec; 1.746 sec/batch)
step 1000, lass=1.38 (71.4 examples/sec; 1.792 sec/batch)
step 1010, lass=1.21 (69.0 examples/sec; 1.856 sec/batch)
step 1020, lass=1.13 (68.5 examples/sec; 1.869 sec/batch)
step 1030, lass=1.18 (55.4 examples/sec; 2.309 sec/batch)
step 1040, lass=1.21 (61.3 examples/sec; 2.089 sec/batch)
step 1050, lass=1.20 (49.6 examples/sec; 2.580 sec/batch)
step 1060, lass=1.20 (53.5 examples/sec; 2.394 sec/batch)
step 1070, lass=1.18 (55.6 examples/sec; 2.301 sec/batch)
step 1080, lass=1.33 (58.4 examples/sec; 2.190 sec/batch)
step 1090, lass=1.28 (63.0 examples/sec; 2.032 sec/batch)
step 1100, lass=1.38 (63.5 examples/sec; 2.016 sec/batch)
step 1110, lass=1.22 (64.6 examples/sec; 1.983 sec/batch)
step 1120, lass=1.46 (64.5 examples/sec; 1.983 sec/batch)
step 1130, lass=1.21 (71.6 examples/sec; 1.787 sec/batch)
step 1140, lass=1.42 (72.4 examples/sec; 1.767 sec/batch)
step 1150, lass=1.20 (73.6 examples/sec; 1.738 sec/batch)
step 1160, lass=1.26 (70.1 examples/sec; 1.827 sec/batch)
step 1170, lass=1.13 (73.2 examples/sec; 1.748 sec/batch)
step 1180, lass=1.28 (68.1 examples/sec; 1.879 sec/batch)
step 1190, lass=1.23 (73.7 examples/sec; 1.737 sec/batch)
step 1200, lass=1.16 (73.5 examples/sec; 1.742 sec/batch)
step 1210, lass=1.17 (68.4 examples/sec; 1.871 sec/batch)
step 1220, lass=1.36 (72.3 examples/sec; 1.771 sec/batch)
step 1230, lass=1.21 (67.8 examples/sec; 1.887 sec/batch)
step 1240, lass=1.21 (67.7 examples/sec; 1.889 sec/batch)
step 1250, lass=1.21 (71.3 examples/sec; 1.795 sec/batch)
step 1260, lass=1.35 (71.2 examples/sec; 1.799 sec/batch)
step 1270, lass=1.22 (69.3 examples/sec; 1.847 sec/batch)
step 1280, lass=1.16 (71.9 examples/sec; 1.781 sec/batch)
step 1290, lass=1.14 (69.0 examples/sec; 1.856 sec/batch)
step 1300, lass=1.22 (72.0 examples/sec; 1.777 sec/batch)
step 1310, lass=1.42 (71.0 examples/sec; 1.803 sec/batch)
step 1320, lass=1.33 (72.0 examples/sec; 1.777 sec/batch)
step 1330, lass=1.30 (72.4 examples/sec; 1.769 sec/batch)
step 1340, lass=1.30 (68.1 examples/sec; 1.881 sec/batch)
step 1350, lass=1.21 (72.0 examples/sec; 1.779 sec/batch)
step 1360, lass=1.04 (72.5 examples/sec; 1.766 sec/batch)
step 1370, lass=1.41 (71.3 examples/sec; 1.796 sec/batch)
step 1380, lass=1.14 (74.2 examples/sec; 1.726 sec/batch)
step 1390, lass=1.35 (69.9 examples/sec; 1.831 sec/batch)
step 1400, lass=1.33 (71.5 examples/sec; 1.790 sec/batch)
step 1410, lass=1.44 (70.0 examples/sec; 1.828 sec/batch)
step 1420, lass=1.15 (70.4 examples/sec; 1.819 sec/batch)
step 1430, lass=1.19 (72.4 examples/sec; 1.769 sec/batch)
step 1440, lass=1.18 (58.2 examples/sec; 2.201 sec/batch)
step 1450, lass=1.30 (56.1 examples/sec; 2.280 sec/batch)
step 1460, lass=1.11 (50.1 examples/sec; 2.557 sec/batch)
step 1470, lass=1.24 (53.7 examples/sec; 2.385 sec/batch)
step 1480, lass=1.52 (50.7 examples/sec; 2.527 sec/batch)
step 1490, lass=1.08 (52.6 examples/sec; 2.433 sec/batch)
step 1500, lass=1.12 (54.7 examples/sec; 2.341 sec/batch)
step 1510, lass=1.11 (51.2 examples/sec; 2.500 sec/batch)
step 1520, lass=1.26 (59.7 examples/sec; 2.142 sec/batch)
step 1530, lass=1.38 (71.7 examples/sec; 1.786 sec/batch)
step 1540, lass=1.17 (71.7 examples/sec; 1.785 sec/batch)
step 1550, lass=1.24 (68.1 examples/sec; 1.879 sec/batch)
step 1560, lass=1.01 (68.9 examples/sec; 1.859 sec/batch)
step 1570, lass=1.23 (70.2 examples/sec; 1.823 sec/batch)
step 1580, lass=1.29 (71.5 examples/sec; 1.789 sec/batch)
step 1590, lass=1.15 (70.0 examples/sec; 1.830 sec/batch)
step 1600, lass=1.18 (69.9 examples/sec; 1.832 sec/batch)
step 1610, lass=1.19 (63.5 examples/sec; 2.016 sec/batch)
step 1620, lass=1.23 (63.8 examples/sec; 2.006 sec/batch)
step 1630, lass=1.21 (54.0 examples/sec; 2.372 sec/batch)
step 1640, lass=1.11 (69.1 examples/sec; 1.853 sec/batch)
step 1650, lass=1.34 (69.9 examples/sec; 1.831 sec/batch)
step 1660, lass=1.10 (73.2 examples/sec; 1.749 sec/batch)
step 1670, lass=1.08 (69.1 examples/sec; 1.852 sec/batch)
step 1680, lass=0.89 (71.5 examples/sec; 1.791 sec/batch)
step 1690, lass=1.28 (69.7 examples/sec; 1.836 sec/batch)
step 1700, lass=1.14 (69.5 examples/sec; 1.842 sec/batch)
step 1710, lass=1.24 (68.6 examples/sec; 1.866 sec/batch)
step 1720, lass=1.16 (72.6 examples/sec; 1.764 sec/batch)
step 1730, lass=1.01 (72.9 examples/sec; 1.757 sec/batch)
step 1740, lass=0.98 (68.0 examples/sec; 1.882 sec/batch)
step 1750, lass=1.16 (72.9 examples/sec; 1.757 sec/batch)
step 1760, lass=1.12 (69.5 examples/sec; 1.842 sec/batch)
step 1770, lass=1.22 (152.5 examples/sec; 0.840 sec/batch)
step 1780, lass=1.03 (149.5 examples/sec; 0.856 sec/batch)
step 1790, lass=1.18 (144.9 examples/sec; 0.884 sec/batch)
step 1800, lass=1.14 (137.0 examples/sec; 0.935 sec/batch)
step 1810, lass=1.38 (123.9 examples/sec; 1.033 sec/batch)
step 1820, lass=1.13 (130.3 examples/sec; 0.983 sec/batch)
step 1830, lass=1.27 (143.0 examples/sec; 0.895 sec/batch)
step 1840, lass=1.20 (150.1 examples/sec; 0.853 sec/batch)
step 1850, lass=1.20 (145.8 examples/sec; 0.878 sec/batch)
step 1860, lass=1.20 (148.7 examples/sec; 0.861 sec/batch)
step 1870, lass=1.41 (146.1 examples/sec; 0.876 sec/batch)
step 1880, lass=0.99 (148.4 examples/sec; 0.863 sec/batch)
step 1890, lass=1.23 (114.3 examples/sec; 1.120 sec/batch)
step 1900, lass=1.13 (144.5 examples/sec; 0.886 sec/batch)
step 1910, lass=1.16 (132.7 examples/sec; 0.965 sec/batch)
step 1920, lass=1.08 (124.0 examples/sec; 1.033 sec/batch)
step 1930, lass=1.20 (129.7 examples/sec; 0.987 sec/batch)
step 1940, lass=1.10 (130.5 examples/sec; 0.981 sec/batch)
step 1950, lass=1.16 (95.9 examples/sec; 1.334 sec/batch)
step 1960, lass=1.37 (125.6 examples/sec; 1.019 sec/batch)
step 1970, lass=1.04 (146.5 examples/sec; 0.874 sec/batch)
step 1980, lass=1.02 (150.4 examples/sec; 0.851 sec/batch)
step 1990, lass=1.26 (128.8 examples/sec; 0.994 sec/batch)
step 2000, lass=1.00 (119.6 examples/sec; 1.070 sec/batch)
step 2010, lass=1.07 (127.2 examples/sec; 1.006 sec/batch)
step 2020, lass=0.98 (128.7 examples/sec; 0.995 sec/batch)
step 2030, lass=1.09 (132.8 examples/sec; 0.964 sec/batch)
step 2040, lass=1.20 (131.1 examples/sec; 0.976 sec/batch)
step 2050, lass=1.38 (128.5 examples/sec; 0.996 sec/batch)
step 2060, lass=1.41 (93.9 examples/sec; 1.363 sec/batch)
step 2070, lass=1.16 (138.8 examples/sec; 0.922 sec/batch)
step 2080, lass=1.24 (98.0 examples/sec; 1.306 sec/batch)
step 2090, lass=1.13 (118.0 examples/sec; 1.085 sec/batch)
step 2100, lass=1.21 (115.4 examples/sec; 1.109 sec/batch)
step 2110, lass=1.15 (130.0 examples/sec; 0.984 sec/batch)
step 2120, lass=1.13 (129.9 examples/sec; 0.985 sec/batch)
step 2130, lass=1.02 (131.2 examples/sec; 0.976 sec/batch)
step 2140, lass=1.07 (114.8 examples/sec; 1.115 sec/batch)
step 2150, lass=1.08 (109.7 examples/sec; 1.166 sec/batch)
step 2160, lass=1.06 (124.7 examples/sec; 1.026 sec/batch)
step 2170, lass=1.11 (123.8 examples/sec; 1.034 sec/batch)
step 2180, lass=1.03 (97.8 examples/sec; 1.309 sec/batch)
step 2190, lass=1.14 (126.2 examples/sec; 1.014 sec/batch)
step 2200, lass=1.13 (121.4 examples/sec; 1.054 sec/batch)
step 2210, lass=1.13 (120.0 examples/sec; 1.067 sec/batch)
step 2220, lass=1.04 (128.4 examples/sec; 0.997 sec/batch)
step 2230, lass=0.88 (115.8 examples/sec; 1.106 sec/batch)
step 2240, lass=1.27 (122.8 examples/sec; 1.043 sec/batch)
step 2250, lass=1.11 (123.8 examples/sec; 1.034 sec/batch)
step 2260, lass=1.28 (116.7 examples/sec; 1.096 sec/batch)
step 2270, lass=1.16 (115.9 examples/sec; 1.105 sec/batch)
step 2280, lass=1.20 (121.8 examples/sec; 1.051 sec/batch)
step 2290, lass=0.97 (127.2 examples/sec; 1.006 sec/batch)
step 2300, lass=1.09 (123.6 examples/sec; 1.035 sec/batch)
step 2310, lass=1.18 (105.4 examples/sec; 1.215 sec/batch)
step 2320, lass=1.03 (129.4 examples/sec; 0.989 sec/batch)
step 2330, lass=0.93 (72.9 examples/sec; 1.756 sec/batch)
step 2340, lass=0.97 (73.2 examples/sec; 1.748 sec/batch)
step 2350, lass=0.92 (65.4 examples/sec; 1.958 sec/batch)
step 2360, lass=1.03 (63.2 examples/sec; 2.025 sec/batch)
step 2370, lass=0.93 (50.6 examples/sec; 2.529 sec/batch)
step 2380, lass=1.18 (64.8 examples/sec; 1.975 sec/batch)
step 2390, lass=1.16 (73.2 examples/sec; 1.748 sec/batch)
step 2400, lass=0.93 (69.5 examples/sec; 1.842 sec/batch)
step 2410, lass=1.09 (67.9 examples/sec; 1.885 sec/batch)
step 2420, lass=1.08 (70.6 examples/sec; 1.813 sec/batch)
step 2430, lass=1.17 (71.6 examples/sec; 1.787 sec/batch)
step 2440, lass=1.05 (69.5 examples/sec; 1.842 sec/batch)
step 2450, lass=1.13 (73.0 examples/sec; 1.754 sec/batch)
step 2460, lass=1.16 (70.1 examples/sec; 1.825 sec/batch)
step 2470, lass=1.14 (72.3 examples/sec; 1.769 sec/batch)
step 2480, lass=1.04 (72.5 examples/sec; 1.767 sec/batch)
step 2490, lass=1.22 (70.5 examples/sec; 1.816 sec/batch)
step 2500, lass=0.96 (72.2 examples/sec; 1.772 sec/batch)
step 2510, lass=1.07 (69.3 examples/sec; 1.846 sec/batch)
step 2520, lass=1.08 (71.9 examples/sec; 1.780 sec/batch)
step 2530, lass=0.98 (71.4 examples/sec; 1.793 sec/batch)
step 2540, lass=1.11 (72.4 examples/sec; 1.767 sec/batch)
step 2550, lass=1.21 (69.6 examples/sec; 1.838 sec/batch)
step 2560, lass=1.05 (70.4 examples/sec; 1.817 sec/batch)
step 2570, lass=0.96 (68.4 examples/sec; 1.870 sec/batch)
step 2580, lass=1.12 (71.5 examples/sec; 1.791 sec/batch)
step 2590, lass=0.92 (71.6 examples/sec; 1.788 sec/batch)
step 2600, lass=1.01 (72.4 examples/sec; 1.767 sec/batch)
step 2610, lass=1.25 (70.5 examples/sec; 1.816 sec/batch)
step 2620, lass=1.13 (69.3 examples/sec; 1.846 sec/batch)
step 2630, lass=1.19 (70.7 examples/sec; 1.811 sec/batch)
step 2640, lass=1.08 (72.0 examples/sec; 1.778 sec/batch)
step 2650, lass=1.09 (66.6 examples/sec; 1.922 sec/batch)
step 2660, lass=1.22 (70.1 examples/sec; 1.826 sec/batch)
step 2670, lass=1.04 (70.5 examples/sec; 1.817 sec/batch)
step 2680, lass=1.08 (67.5 examples/sec; 1.895 sec/batch)
step 2690, lass=1.12 (67.7 examples/sec; 1.890 sec/batch)
step 2700, lass=0.84 (72.4 examples/sec; 1.767 sec/batch)
step 2710, lass=0.85 (71.3 examples/sec; 1.794 sec/batch)
step 2720, lass=1.18 (71.6 examples/sec; 1.789 sec/batch)
step 2730, lass=1.11 (70.8 examples/sec; 1.807 sec/batch)
step 2740, lass=1.14 (68.4 examples/sec; 1.871 sec/batch)
step 2750, lass=0.97 (73.5 examples/sec; 1.741 sec/batch)
step 2760, lass=1.06 (72.7 examples/sec; 1.760 sec/batch)
step 2770, lass=1.02 (71.6 examples/sec; 1.787 sec/batch)
step 2780, lass=1.03 (71.4 examples/sec; 1.794 sec/batch)
step 2790, lass=1.34 (68.9 examples/sec; 1.858 sec/batch)
step 2800, lass=1.06 (72.5 examples/sec; 1.766 sec/batch)
step 2810, lass=1.19 (72.6 examples/sec; 1.763 sec/batch)
step 2820, lass=0.96 (70.8 examples/sec; 1.809 sec/batch)
step 2830, lass=1.08 (73.0 examples/sec; 1.753 sec/batch)
step 2840, lass=0.93 (70.0 examples/sec; 1.829 sec/batch)
step 2850, lass=0.89 (73.1 examples/sec; 1.751 sec/batch)
step 2860, lass=1.13 (70.6 examples/sec; 1.814 sec/batch)
step 2870, lass=1.26 (72.7 examples/sec; 1.762 sec/batch)
step 2880, lass=1.13 (74.3 examples/sec; 1.723 sec/batch)
step 2890, lass=0.97 (67.9 examples/sec; 1.886 sec/batch)
step 2900, lass=0.90 (74.5 examples/sec; 1.719 sec/batch)
step 2910, lass=1.20 (67.8 examples/sec; 1.887 sec/batch)
step 2920, lass=0.99 (71.6 examples/sec; 1.787 sec/batch)
step 2930, lass=1.14 (74.3 examples/sec; 1.723 sec/batch)
step 2940, lass=1.08 (70.4 examples/sec; 1.818 sec/batch)
step 2950, lass=1.12 (73.2 examples/sec; 1.748 sec/batch)
step 2960, lass=1.06 (71.4 examples/sec; 1.793 sec/batch)
step 2970, lass=1.21 (68.3 examples/sec; 1.875 sec/batch)
step 2980, lass=1.05 (72.1 examples/sec; 1.775 sec/batch)
step 2990, lass=1.10 (73.3 examples/sec; 1.746 sec/batch)
precision @ 1 = 0.000