tensorflow课堂笔记（五）

最新推荐文章于 2020-03-22 11:34:56 发布

haohulala

最新推荐文章于 2020-03-22 11:34:56 发布

阅读量356

点赞数

分类专栏： TensorFlow 文章标签： tensorflow

本文链接：https://blog.csdn.net/haohulala/article/details/83653912

版权

TensorFlow 专栏收录该内容

7 篇文章 0 订阅

订阅专栏

正则化

"""
正则化缓解过拟合
正则化在损失函数中引入模型复杂度指标，利用给W加权值，弱化了训练数据的噪声
loss=loss(y,y_)+REGULAERIZER*loss(w)
loss(w)=tf.contrib.layers.l1_regularizer(REGULAERIZER)(w) lossL1(w1)=∑|wi|
loss(w)=tf.contrib.layers.l2_regularizer(REGULAERIZER)(w) lossL2(w2)=∑|wi²|
tf.add_to_collection('losses',tf.contrib.layers.l2_regularizer(regularizer)(w))
loss=cem+tf.add_n(tf.get_collection('losses'))
"""
#coding utf-8
#0导入模块，生成模拟数据集
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
BATCH_SIZE = 30
seed = 2
#基于seed产生随机数
rdm = np.random.RandomState(seed)
#生虫300组2列数据作为输入集
X = rdm.randn(300,2)     #n-normal返回一组具有正态分布的样本
#从数据集中取出一行，如果平方和小于等于2 Y为1，否则为0
Y_ = [int(x0*x0 + x1*x1 < 2) for [x0,x1] in X]
#遍历Y中的每个元素，1赋值‘red’，0赋值‘blue’
Y_c = [['red' if y else 'blue']for y in Y_]
#对数据集X和标签Y进行shape整理，第一个参数为-1表示任意实数，将X整理成n行2列，Y整理成n行1列
X = np.vstack(X).reshape(-1, 2)
Y_ = np.vstack(Y_).reshape(-1, 1)
print(X)
print(Y_)
print(Y_c)
#用plt.scatter画出数据集X各行中第0列元素和第1列元素的点，及各行的（X0, X1），用各行Y_c对应的值表示颜色
plt.scatter(X[:,0],X[:,1],c=np.squeeze(Y_c))   #染色
plt.show()

#定义神经网络的输入，参数和输出，定义前向传播过程
def get_weight(shape, regularizer):
    w = tf.Variable(tf.random_normal(shape), dtype=tf.float32)
    tf.add_to_collection('losses', tf.contrib.layers.l2_regularizer(regularizer)(w))
    return w

def get_bias(shape):
    b = tf.Variable(tf.constant(0.01, shape=shape))
    return b

x = tf.placeholder(tf.float32, shape=[None, 2])
y_ = tf.placeholder(tf.float32, shape=[None, 1])

w1 = get_weight([2,11], 0.01)
b1 = get_bias([11])
y1 = tf.nn.relu(tf.matmul(x, w1)+b1)

w2 = get_weight([11,1], 0.01)
b2 = get_bias([1])
y = tf.matmul(y1, w2)+b2           #输出层不激活

#定义损失函数
loss_mse = tf.reduce_mean(tf.square(y-y_))
loss_total = loss_mse + tf.add_n(tf.get_collection('losses'))

#定义反向传播方法：不含正则化
train_step = tf.train.AdamOptimizer(0.0001).minimize(loss_mse)

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)
    STEPS = 40000
    for i in range(STEPS):
        start = (i % BATCH_SIZE) % 300
        end = start + BATCH_SIZE
        sess.run(train_step, feed_dict={x:X[start:end],y_:Y_[start:end]})
        if i % 2000 == 0:
            loss_mse_v = sess.run(loss_mse,feed_dict={x:X, y_:Y_})
            print("After %d steps, loss is : %f"%(i, loss_mse_v))
    #xx在-3到3之间以步长为0.01， yy在-3到3之间以步长0.01生成二维网络坐标点
    xx, yy = np.mgrid[-3:3:.01, -3:3:.01]
    #将xx, yy拉直，并合并成一个2列的矩阵，得到一个网格坐标点的集合
    grid = np.c_[xx.ravel(), yy.ravel()]
    #将网格坐标点喂入神经网络，probs为输出
    probs = sess.run(y, feed_dict={x:grid})
    #probs的shape调整成xx的样子
    probs = probs.reshape(xx.shape)
    print("w1:\n", sess.run(w1))
    print("b1:\n", sess.run(b1))
    print("w2:\n", sess.run(w2))

plt.scatter(X[:,0],X[:,1],c = np.squeeze(Y_c))
plt.contour(xx, yy ,probs, levels=[.5])
plt.show()

#定义反向传播方法：包含正则化
train_step = tf.train.AdamOptimizer(0.0001).minimize(loss_total)

with tf.Session() as sess:
    init_op = tf.global_variables_initializer()
    sess.run(init_op)
    STEPS = 40000
    for i in range(STEPS):
        start = (i%BATCH_SIZE) % 300
        end = start + BATCH_SIZE
        sess.run(train_step, feed_dict={x: X[start:end],y_ : Y_[start:end]})
        if i % 2000 == 0:
            loss_v = sess.run(loss_total, feed_dict={x: X, y_:Y_})
            print("After %d steps, loss is : %f"%(i,loss_v))

    xx, yy = np.mgrid[-3:3:.01, -3:3:.01]
    grid = np.c_[xx.ravel(), yy.ravel()]
    probs = sess.run(y, feed_dict={x:grid})
    probs = probs.reshape(xx.shape)
    print("w1:\n", sess.run(w1))
    print("b1:\n", sess.run(b1))
    print("w2:\n", sess.run(w2))
    print("b2:\n", sess.run(b2))

plt.scatter(X[:,0],X[:,1], c = np.squeeze(Y_c))
plt.contour(xx, yy, probs, levels = [.5])
plt.show()

下面是输出结果：

数据集

没有正则化的结果

After 0 steps, loss is : 3.268635
After 2000 steps, loss is : 0.388300
After 4000 steps, loss is : 0.190906
After 6000 steps, loss is : 0.126302
After 8000 steps, loss is : 0.109097
After 10000 steps, loss is : 0.102286
After 12000 steps, loss is : 0.100818
After 14000 steps, loss is : 0.099644
After 16000 steps, loss is : 0.099715
After 18000 steps, loss is : 0.100641
After 20000 steps, loss is : 0.100605
After 22000 steps, loss is : 0.101486
After 24000 steps, loss is : 0.104060
After 26000 steps, loss is : 0.107447
After 28000 steps, loss is : 0.120574
After 30000 steps, loss is : 0.135065
After 32000 steps, loss is : 0.146016
After 34000 steps, loss is : 0.153508
After 36000 steps, loss is : 0.157925
After 38000 steps, loss is : 0.159713
w1:
 [[ 0.41214204  1.3394859  -0.02082919 -0.033639   -0.9077819   0.3915897
   0.42436182 -0.07539023  0.979067   -1.5770481  -0.24494807]
 [-1.2968411   0.3786142  -0.22384964 -0.12804544 -0.3873543  -0.88351375
  -1.1493685  -0.25389695  0.0335511  -0.23689292 -2.4217346 ]]
b1:
 [-2.6347706e-01 -1.0653939e+00  4.9396348e-01  2.7674437e-01
 -5.3983215e-02 -2.5115678e-01 -2.0593126e-01  5.4631197e-01
  1.8800445e-01 -6.6704006e-04 -4.3486357e-01]
w2:
 [[ 0.1192633 ]
 [-0.8521139 ]
 [ 0.8938104 ]
 [ 1.0471085 ]
 [-1.0349886 ]
 [-1.4969603 ]
 [ 0.03636099]
 [ 0.59861106]
 [ 0.27789012]
 [ 0.18401219]
 [ 0.0409697 ]]

正则化的结果：

After 0 steps, loss is : 36.077763
After 2000 steps, loss is : 10.777819
After 4000 steps, loss is : 3.060398
After 6000 steps, loss is : 1.058339
After 8000 steps, loss is : 0.599815
After 10000 steps, loss is : 0.448360
After 12000 steps, loss is : 0.368117
After 14000 steps, loss is : 0.321465
After 16000 steps, loss is : 0.268670
After 18000 steps, loss is : 0.226850
After 20000 steps, loss is : 0.195428
After 22000 steps, loss is : 0.175436
After 24000 steps, loss is : 0.161367
After 26000 steps, loss is : 0.148957
After 28000 steps, loss is : 0.140198
After 30000 steps, loss is : 0.136005
After 32000 steps, loss is : 0.133354
After 34000 steps, loss is : 0.131369
After 36000 steps, loss is : 0.129640
After 38000 steps, loss is : 0.129228
w1:
 [[-3.7225670e-01  4.3736741e-01  8.4175813e-01  4.4566866e-33
  -3.3428264e-01  9.5313505e-33 -4.5814252e-01  6.3367188e-02
  -2.0819646e-01 -7.4419081e-02  4.1060110e-33]
 [ 7.4761458e-02  3.9469945e-01  1.1804464e-01  4.7707457e-33
  -5.1289779e-01  3.2189234e-33  3.0285943e-01  8.0356359e-01
   2.7540690e-01  3.4937870e-02 -5.3322444e-33]]
b1:
 [-3.0600670e-01  9.6624523e-01 -5.1496935e-01 -1.9298960e-01
 -2.4860874e-01 -7.0885391e-05 -2.0954745e-01 -3.1623110e-01
  7.2149241e-01  1.3185053e-01 -3.9324257e-01]
w2:
 [[-3.5867396e-01]
 [ 2.9064563e-01]
 [-9.6794862e-01]
 [-1.5731245e-04]
 [-6.4135724e-01]
 [-9.5987100e-33]
 [-5.1885021e-01]
 [-8.1023085e-01]
 [ 4.5911625e-01]
 [ 9.8131806e-02]
 [-4.1043961e-33]]
b2:
 [0.4127569]

正则化后的收敛效果要好一点

haohulala

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
2
评论
tensorflow课堂笔记（五）

正则化"""正则化缓解过拟合正则化在损失函数中引入模型复杂度指标，利用给W加权值，弱化了训练数据的噪声loss=loss(y,y_)+REGULAERIZER*loss(w)loss(w)=tf.contrib.layers.l1_regularizer(REGULAERIZER)(w) lossL1(w1)=∑|wi|loss(w)=tf.contrib.layers.l2_re...
复制链接

扫一扫