tensorflow笔记实践：正则化优化过拟合

学习一下曹健老师的视频，老师辛苦了。

主要介绍用正则化方法来防止过拟合，想学习的同学可以在这里观看学习：https://www.icourse163.org/learn/PKU-1002536002?tid=1002700003#/learn/content?type=detail&id=1004043074&sm=1

我是在windows上用pycharm（python3.6）做的，跟老师有点不一样不过没关系。代码如下：

import tensorflow as tf
import numpy as np
import  matplotlib.pyplot as plt
# 数据，参数
seed = 2
batch_size = 30
rdm = np.random.RandomState(seed)
X = rdm.randn(300,2)
Y_ = [int(X0*X0+X1*X1<2)for(X0,X1)in X]
Y_c = [['red'if y else 'blue']for y in Y_]
X = np.vstack(X).reshape(-1,2)
Y_ = np.vstack(Y_).reshape(-1,1)
#显示
#参数定义，前向传播
x_data = tf.placeholder(dtype=tf.float32,shape=[None,2])
y_data = tf.placeholder(dtype=tf.float32,shape=[None,1])
def weight_fuc(shape,reg):
    w = tf.Variable(tf.random_normal(shape),dtype=tf.float32)
    tf.add_to_collection('losses',tf.contrib.layers.l2_regularizer(reg)(w))
    return w
def bais_fuc(shape):
    b = tf.Variable(tf.constant(0.01,shape=shape))
    return b
w1 =weight_fuc([2,11],0.01)
b1 = bais_fuc([11])
y1 = tf.nn.relu(tf.matmul(x_data,w1)+b1)
w2 = weight_fuc([11,1],0.01)
b2 = bais_fuc([1])
y2 = tf.matmul(y1,w2)+b2
#定义损失函数
loss = tf.reduce_mean(tf.square(y2-y_data))
# loss_total = loss +tf.add_n(tf.get_collection('losses'))
train_step = tf.train.GradientDescentOptimizer(0.01).minimize(loss)
with tf.Session() as sess:
    init = tf.global_variables_initializer()
    sess.run(init)
    for i in range(40000):
        start = (i*batch_size) % 300
        end = start + batch_size
        sess.run(train_step,feed_dict={x_data:X[start:end],y_data:Y_[start:end]})
    xx,yy = np.mgrid[-3:3:.01,-3:3:.01]
    grid = np.c_[xx.ravel(),yy.ravel()]
    probs = sess.run(y2,feed_dict={x_data:grid})
    probs = probs.reshape(xx.shape)
plt.scatter(X[:,0],X[:,1],c=np.squeeze(Y_c))
plt.contour(xx,yy,probs,levels=[.5])
plt.show()

 不加入正则化的效果如下图：

 正则化很简单，只要改一下损失函数（代码中注释掉的），把损失函数改成loss_total加入正则化的效果如下图：

总体看来加入正则化项后得到曲线没有那么尖锐，变得圆滑一些。