【tensorflow 学习】给LSTM加上L2正则化

Reference
https://stackoverflow.com/questions/41841050/tensorflow-adding-regularization-to-lstm?noredirect=1&lq=1

1. 首先，用tf.trainable_variables()得到所有weights和bias，
2. 然后，用tf.nn.l2_loss()计算L2 norm，
3. 求和之后作为正则项加给原来的cost function

tv = tf.trainable_variables()#得到所有可以训练的参数，即所有trainable=True 的tf.Variable/tf.get_variable
regularization_cost = 0.001* tf.reduce_sum([ tf.nn.l2_loss(v) for v in tv ]) #0.001是lambda超参数
cost = original_cost_function + regularization_cost
optimizer = tf.train.AdamOptimizer(learning_rate = 0.01).minimize(cost)