python权重矩阵_python – 使用TensorFlow重整化权重矩阵

这是一个可能的实现：

import tensorflow as tf

def maxnorm_regularizer(threshold,axes=1,name="maxnorm",collection="maxnorm"):

def maxnorm(weights):

clipped = tf.clip_by_norm(weights,clip_norm=threshold,axes=axes)

clip_weights = tf.assign(weights,clipped,name=name)

tf.add_to_collection(collection,clip_weights)

return None # there is no regularization loss term

return maxnorm

以下是您将如何使用它：

from tensorflow.contrib.layers import fully_connected

from tensorflow.contrib.framework import arg_scope

with arg_scope(

[fully_connected],weights_regularizer=max_norm_regularizer(1.5)):

hidden1 = fully_connected(X,200,scope="hidden1")

hidden2 = fully_connected(hidden1,100,scope="hidden2")

outputs = fully_connected(hidden2,5,activation_fn=None,scope="outs")

max_norm_ops = tf.get_collection("max_norm")

[...]

with tf.Session() as sess:

sess.run(init)

for epoch in range(n_epochs):

for X_batch,y_batch in load_next_batch():

sess.run(training_op,feed_dict={X: X_batch,y: y_batch})

sess.run(max_norm_ops)

这将创建一个3层神经网络,并在每一层(阈值为1.5)对其进行最大范数正则化训练.我只是尝试过,似乎工作.希望这可以帮助！欢迎提出改进建议.