Tensorflow(三十二) —— 梯度爆炸
解决方案
import tensorflow as tf
from tensorflow import keras
# solve gradient exploding
with tf.GradientTape() as tape:
logits = model(x)
loss = tf.losses.categorical_crossentropy(y,logits,from_logits=True)
grads = tape.gradient(loss,model.trainable_variables)
grads = [tf.clip_by_norm(g,15) for g in grads]
optimizer.apply_gradients(zip(grads,model.trainable_variables))
本文为参考龙龙老师的“深度学习与TensorFlow 2入门实战“课程书写的学习笔记
by CyrusMay 2022 04 16