3-Tensorflow-demo_11-学习率的衰减

最新推荐文章于 2021-03-17 14:47:24 发布

HJZ11

最新推荐文章于 2021-03-17 14:47:24 发布

阅读量140

点赞数

分类专栏： # 深度学习3-Tensorflow

本文链接：https://blog.csdn.net/HJZ11/article/details/103413120

版权

深度学习3-Tensorflow 专栏收录该内容

33 篇文章 1 订阅

订阅专栏

tf.train.exponential_decay(
    learning_rate,初始学习率
    global_step,当前迭代次数
    decay_steps,衰减速度（在迭代到该次数时学习率衰减为earning_rate * decay_rate）
    decay_rate,学习率衰减系数，通常介于0-1之间。
    staircase=False,(默认值为False,当为True时，（global_step/decay_steps）则被转化为整数) ,选择不同的衰减方式。
    name=None

import tensorflow as tf
import os
import numpy as np

x = tf.placeholder(tf.float32, [None, 1], name='x')
y = tf.placeholder(tf.float32, [None, 1], name='y')
w = tf.Variable(initial_value=tf.constant(0.0))

# 定义一个计算步数的变量
global_steps = tf.Variable(initial_value=0, trainable=False)
# global_steps = tf.train.get_or_create_global_step()

learning_rate = tf.train.exponential_decay(
    learning_rate=0.1,
    global_step=global_steps,
    decay_steps=10,
    decay_rate=0.9
)

# 定义损失
loss = tf.pow(w*x - y, 2)

train_opt = tf.train.GradientDescentOptimizer(learning_rate=learning_rate).\
    minimize(loss, global_step=global_steps)

# 构建会话
epochs = 3
batch_cnt = 10

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    for e in range(epochs):
        for batch in range(batch_cnt):
            feed = {x: np.linspace(1, 2, 10).reshape([10, 1]),
                    y: np.linspace(1, 2, 10).reshape([10, 1])}
            sess.run(train_opt, feed)
            print(sess.run(learning_rate))
            print(sess.run(global_steps))

D:\Anaconda\python.exe D:/AI20/HJZ/04-深度学习/2-TensorFlow基础/tf-代码/11_学习率的衰减.py
2019-12-05 21:46:32.379345: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX AVX2
0.09895193
1
0.09791484
2
0.09688862
3
0.095873155
4
0.094868325
5
0.09387404
6
0.092890166
7
0.09191661
8
0.09095325
9
0.089999996
10
0.08905673
11
0.08812335
12
0.087199755
13
0.08628584
14
0.0853815
15
0.08448663
16
0.08360115
17
0.08272495
18
0.08185793
19
0.08099999
20
0.08015105
21
0.07931101
22
0.078479774
23
0.077657245
24
0.07684334
25
0.076037966
26
0.07524103
27
0.07445245
28
0.07367214
29
0.07289999
30

Process finished with exit code 0