keras 自定义embed layer,如下:
class ConcatClassTokenAddPositionEmbedding(layers.Layer):
def __init__(self, embed_dims, token_num, name='None'):
super(ConcatClassTokenAddPositionEmbedding, self).__init__(name=name)
self.embed_dims = embed_dims
self.token_num = token_num
def build(self, input_shape):
self.class_token = self.add_weight(
shape=[1, 1, self.embed_dims],
initializer=keras.initializers.initializers_v2.Zeros(),
trainable=True,
dtype=tf.float32)
self.w = self.add_weight(
shape=[1, input_shape[1] + 1, input_shape[2]],
initializer=keras.initializers.initializers_v2.HeUniform(),
trainable=True)
在使用model.save('model.h5')保存模型时,崩溃,错误提示如下:
dset_id = h5d.create(parent.id, name, tid, sid, dcpl=dcpl)
File "h5py\_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
File "h5py\_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
File "h5py\h5d.pyx", line 87, in h5py.h5d.create
ValueError: Unable to create dataset (name already exists)
问题原因:模型中有变量重名(name already exists)。具体的是自定义层中定义了两个可训练变量,即self.class_token和 self.w。并且在定义时,self.class_token = self.add_weight()没有指定name参数,name参数为空时,使用默认名字:Variable.因为self.class_token和slef.w都没有指定name,所以二者名字都是Variable,导致重名。
解决方法:
add_weight中设置不同的name参数即可,即:
self.class_token = self.add_weight(name='cls_token',
shape=[1, 1, self.embed_dims],
initializer=keras.initializers.initializers_v2.Zeros(),
trainable=True,
dtype=tf.float32)
print('class_token' + str(self.class_token.shape))
self.w = self.add_weight(name='pos_embedding',
shape=[1, input_shape[1] + 1, input_shape[2]],
initializer=keras.initializers.initializers_v2.HeUniform(),
trainable=True)
问题解决!