tensorflow函数中minimize()函数

使用方法

opt = tf.tf.train.GradientDescentOptimizer(learning_rate=0.1)
train_op = opt.minimize(loss)

minimize的内部存在两个操作：(1)计算各个变量的梯度 (2)用梯度更新这些变量的值

主要的参数说明：

      loss:  `Tensor` ，需要优化的损失； 
      var_list: 需要更新的变量(tf.Varialble)组成的列表或者元组，默认值为`GraphKeys.TRAINABLE_VARIABLES`，即tf.trainable_variables()
注意：

1、Optimizer.minimize(loss, var_list)中，计算loss所涉及的变量(假设为var(loss))包含在var_list中，也就是var_list中含有多余的变量，并不 影响程序的运行，而且优化过程中不改变var_list里多出变量的值；

2、若var_list中的变量个数少于var(loss)，则优化过程中只会更新var_list中的那些变量的值，var(loss)里多出的变量值 并不会改变，相当于固定了网络的某一部分的参数值。

 

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    #在for循环内进行30000训练
    for i in range(training_steps):
        sess.run(train_op, feed_dict={x: data, y_: label})
        #训练30000轮，但每隔2000轮就输出一次loss的值
        if i % 2000 == 0:
            loss_value = sess.run(loss, feed_dict={x: data, y_: label})
            print("After %d steps, loss_value is: %f" % (i,loss_value))

下面是将上面加蓝色代码注释掉运行的结果如下

After 0 steps, loss_value is: 0.512605
After 2000 steps, loss_value is: 0.512605
After 4000 steps, loss_value is: 0.512605
After 6000 steps, loss_value is: 0.512605
After 8000 steps, loss_value is: 0.512605
After 10000 steps, loss_value is: 0.512605
After 12000 steps, loss_value is: 0.512605
After 14000 steps, loss_value is: 0.512605
After 16000 steps, loss_value is: 0.512605
After 18000 steps, loss_value is: 0.512605
After 20000 steps, loss_value is: 0.512605
After 22000 steps, loss_value is: 0.512605
After 24000 steps, loss_value is: 0.512605
After 26000 steps, loss_value is: 0.512605
After 28000 steps, loss_value is: 0.512605

 下面是没注释的完整代码跑出来的结果

import tensorflow as tf
import  numpy as np
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"
#定义训练轮数
training_steps = 30000

#定义输入的数据和对应的标签并在for循环内进行填充
data = []
label = []
for i in range(200):
    x1 = np.random.uniform(-1, 1)
    x2 = np.random.uniform(0, 2)
    #这里对产生的x1和x2进行判断，如果产生的点落在半径为1的圆内，则label值为0
    #否则label值为1
    if x1**2 + x2**2 <= 1:
        data.append([np.random.normal(x1, 0.1),np.random.normal(x2,0.1)])
        label.append(0)
    else:
        data.append([np.random.normal(x1, 0.1), np.random.normal(x2, 0.1)])
        label.append(1)

#numpy的hstack()函数用于在水平方向将元素堆起来
#函数原型numpy.hstack(tup)，参数tup可以是元组、列表、或者numpy数组，
#返回结果为numpy的数组。reshape()函数的参数-1表示行列进行翻转。
#这样处理的结果为data变成了200x2大小的数组，而label是100x1
data = np.hstack(data).reshape(-1,2)
label = np.hstack(label).reshape(-1, 1)

#定义完成前向传播的隐层
def hidden_layer(input_tensor,weight1,bias1,weight2,bias2,weight3,bias3):
    layer1=tf.nn.relu(tf.matmul(input_tensor,weight1)+bias1)
    layer2 = tf.nn.relu(tf.matmul(layer1, weight2) + bias2)
    return tf.matmul(layer2,weight3)+bias3


x = tf.placeholder(tf.float32, shape=(None, 2),name="x-input")
y_ = tf.placeholder(tf.float32, shape=(None, 1),name="y-output")

#定义权重参数和偏置参数
weight1 = tf.Variable(tf.truncated_normal([2,10],stddev=0.1))
bias1 = tf.Variable(tf.constant(0.1, shape=[10]))
weight2 = tf.Variable(tf.truncated_normal([10,10],stddev=0.1))
bias2 = tf.Variable(tf.constant(0.1, shape=[10]))
weight3 = tf.Variable(tf.truncated_normal([10,1],stddev=0.1))
bias3 = tf.Variable(tf.constant(0.1, shape=[1]))

#用len()函数计算data数组的长度
sample_size = len(data)

#得到隐层前向传播结果
y= hidden_layer(x,weight1,bias1,weight2,bias2,weight3,bias3)

#自定义的损失函数。pow()函数用于计算幂函数，原型为pow(x,y,name=None)
#返回结果为x的y次幂，这里返回结果为(y_-y)^2用于衡量计算值与实际值的差距
error_loss = tf.reduce_sum(tf.pow(y_ - y, 2)) / sample_size
tf.add_to_collection("losses", error_loss)      #加入集合的操作

#在权重参数上实现L2正则化
regularizer = tf.contrib.layers.l2_regularizer(0.01)
regularization = regularizer(weight1)+regularizer(weight2)+regularizer(weight3)
tf.add_to_collection("losses",regularization)     #加入集合的操作

#get_collection()函数获取指定集合中的所有个体，这里是获取所有损失值
#并在add_n()函数中进行加和运算
loss = tf.add_n(tf.get_collection("losses"))

#定义一个优化器，学习率为固定为0.01，注意在实际应用中这个学习率数值应该大于0.01
train_op = tf.train.AdamOptimizer(0.01).minimize(loss)

with tf.Session() as sess:
    tf.global_variables_initializer().run()
    #在for循环内进行30000训练
    for i in range(training_steps):
        sess.run(train_op, feed_dict={x: data, y_: label})
        #训练30000轮，但每隔2000轮就输出一次loss的值
        if i % 2000 == 0:
            loss_value = sess.run(loss, feed_dict={x: data, y_: label})
            print("After %d steps, loss_value is: %f" % (i,loss_value))

结果如下：

After 0 steps, loss_value is: 0.403034
After 2000 steps, loss_value is: 0.071240
After 4000 steps, loss_value is: 0.071202
After 6000 steps, loss_value is: 0.071197
After 8000 steps, loss_value is: 0.071181
After 10000 steps, loss_value is: 0.070808
After 12000 steps, loss_value is: 0.070798
After 14000 steps, loss_value is: 0.070798
After 16000 steps, loss_value is: 0.070808
After 18000 steps, loss_value is: 0.070803
After 20000 steps, loss_value is: 0.070796
After 22000 steps, loss_value is: 0.070794
After 24000 steps, loss_value is: 0.070800
After 26000 steps, loss_value is: 0.070800
After 28000 steps, loss_value is: 0.070800

Process finished with exit code 0

参考博客https://blog.csdn.net/weixin_40597170/article/details/96012213

https://blog.csdn.net/Huang_Fj/article/details/102688509