花呗问答匹配（part3 各种模型运行）

当前时间是2021/6/16，matchzoo作为非常强大的文本匹配库，当前未更新到TF2.4以上版本，无法使用本机3090加速，为此我将源码反向推导实现，使用TF2.4形式实现这些模型

"""
作者英俊
QQ 2227495940
所有权：西安建筑科技大学草堂校区 信控楼704实验室
"""
"暂定只能扣13个模型出来"

'暂定只能扣13个模型出来'

## Denseline


```python
input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
# embedding = Embedding(vocab_size, embedding_size)
# sent1_embed=embedding(input1)
# sent2_embed=embedding(input2)
concatenate_1=concatenate([input1,input2],axis=-1)
dense1=Dense(300)(concatenate_1)
dense2=Dense(300)(dense1)
dense3=Dense(128)(dense2)
outputs=Dense(1,activation='linear',name="output")(dense3)
model=Model(inputs=[input1,input2],outputs=outputs,name="Denseline")
model.summary()

Model: "Denseline"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 30)           0           text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
dense (Dense)                   (None, 300)          9300        concatenate[0][0]                
__________________________________________________________________________________________________
dense_1 (Dense)                 (None, 300)          90300       dense[0][0]                      
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 128)          38528       dense_1[0][0]                    
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            129         dense_2[0][0]                    
==================================================================================================
Total params: 138,257
Trainable params: 138,257
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 2s 15ms/step - loss: 3.2004 - accuracy: 0.7902 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00001: val_accuracy improved from -inf to 0.81637, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 4ms/step - loss: 3.2150 - accuracy: 0.7916 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00002: val_accuracy did not improve from 0.81637
Epoch 3/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9334 - accuracy: 0.8098 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00003: val_accuracy did not improve from 0.81637
Epoch 4/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1602 - accuracy: 0.7951 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00004: val_accuracy did not improve from 0.81637
Epoch 5/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1563 - accuracy: 0.7954 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00005: val_accuracy did not improve from 0.81637
Epoch 6/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7483 - accuracy: 0.8218 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00006: val_accuracy did not improve from 0.81637
Epoch 7/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9518 - accuracy: 0.8086 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00007: val_accuracy did not improve from 0.81637
Epoch 8/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0769 - accuracy: 0.8005 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00008: val_accuracy did not improve from 0.81637
Epoch 9/50
63/63 [==============================] - ETA: 0s - loss: 2.9823 - accuracy: 0.80 - 0s 4ms/step - loss: 2.9761 - accuracy: 0.8071 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00009: val_accuracy did not improve from 0.81637
Epoch 10/50
63/63 [==============================] - 0s 4ms/step - loss: 2.6330 - accuracy: 0.8293 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00010: val_accuracy did not improve from 0.81637
Epoch 11/50
63/63 [==============================] - 0s 4ms/step - loss: 3.2002 - accuracy: 0.7925 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00011: val_accuracy did not improve from 0.81637
Epoch 12/50
63/63 [==============================] - 0s 6ms/step - loss: 2.7750 - accuracy: 0.8201 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00012: val_accuracy did not improve from 0.81637
Epoch 13/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1518 - accuracy: 0.7957 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00013: val_accuracy did not improve from 0.81637
Epoch 14/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0377 - accuracy: 0.8031 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00014: val_accuracy did not improve from 0.81637
Epoch 15/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9777 - accuracy: 0.8070 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00015: val_accuracy did not improve from 0.81637
Epoch 16/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9072 - accuracy: 0.8115 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00016: val_accuracy did not improve from 0.81637
Epoch 17/50
63/63 [==============================] - 0s 4ms/step - loss: 2.6993 - accuracy: 0.8250 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00017: val_accuracy did not improve from 0.81637
Epoch 18/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1956 - accuracy: 0.7928 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00018: val_accuracy did not improve from 0.81637
Epoch 19/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1464 - accuracy: 0.7960 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00019: val_accuracy did not improve from 0.81637
Epoch 20/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0472 - accuracy: 0.8024 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00020: val_accuracy did not improve from 0.81637
Epoch 21/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9581 - accuracy: 0.8082 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00021: val_accuracy did not improve from 0.81637
Epoch 22/50
63/63 [==============================] - 0s 4ms/step - loss: 3.1285 - accuracy: 0.7972 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00022: val_accuracy did not improve from 0.81637
Epoch 23/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0444 - accuracy: 0.8026 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00023: val_accuracy did not improve from 0.81637
Epoch 24/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0231 - accuracy: 0.8040 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00024: val_accuracy did not improve from 0.81637
Epoch 25/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0962 - accuracy: 0.7993 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00025: val_accuracy did not improve from 0.81637
Epoch 26/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9691 - accuracy: 0.8075 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00026: val_accuracy did not improve from 0.81637
Epoch 27/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7456 - accuracy: 0.8220 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00027: val_accuracy did not improve from 0.81637
Epoch 28/50
63/63 [==============================] - 0s 4ms/step - loss: 3.0548 - accuracy: 0.8020 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00028: val_accuracy did not improve from 0.81637
Epoch 29/50
63/63 [==============================] - 0s 4ms/step - loss: 2.8902 - accuracy: 0.8126 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00029: val_accuracy did not improve from 0.81637
Epoch 30/50
63/63 [==============================] - 0s 4ms/step - loss: 2.8348 - accuracy: 0.8162 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00030: val_accuracy did not improve from 0.81637
Epoch 31/50
63/63 [==============================] - 0s 4ms/step - loss: 2.7539 - accuracy: 0.8215 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00031: val_accuracy did not improve from 0.81637
Epoch 32/50
63/63 [==============================] - 0s 4ms/step - loss: 2.9578 - accuracy: 0.8082 - val_loss: 2.8325 - val_accuracy: 0.8164

Epoch 00032: val_accuracy did not improve from 0.81637
Denseline运行时间: 0.21 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\save\Denseline\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Denseline\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QeHYE8H6-1623898674919)(output_41_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2WMSiOqS-1623898674920)(output_41_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	3.007824	0.8045	2.832526	0.816367	0.0010
2	2.984728	0.8065	2.832526	0.816367	0.0010
3	2.984727	0.8065	2.832526	0.816367	0.0010
4	2.984727	0.8065	2.832526	0.816367	0.0010
5	2.984728	0.8065	2.832526	0.816367	0.0010
6	2.984727	0.8065	2.832526	0.816367	0.0010
7	2.984727	0.8065	2.832526	0.816367	0.0010
8	2.984727	0.8065	2.832526	0.816367	0.0010
9	2.984727	0.8065	2.832526	0.816367	0.0010
10	2.984728	0.8065	2.832526	0.816367	0.0010
11	2.984727	0.8065	2.832526	0.816367	0.0010
12	2.984728	0.8065	2.832526	0.816367	0.0010
13	2.984727	0.8065	2.832526	0.816367	0.0010
14	2.984727	0.8065	2.832526	0.816367	0.0010
15	2.984727	0.8065	2.832526	0.816367	0.0010
16	2.984728	0.8065	2.832526	0.816367	0.0010
17	2.984727	0.8065	2.832526	0.816367	0.0010
18	2.984727	0.8065	2.832526	0.816367	0.0010
19	2.984728	0.8065	2.832526	0.816367	0.0010
20	2.984727	0.8065	2.832526	0.816367	0.0010
21	2.984728	0.8065	2.832526	0.816367	0.0010
22	2.984728	0.8065	2.832526	0.816367	0.0010
23	2.984727	0.8065	2.832526	0.816367	0.0010
24	2.984728	0.8065	2.832526	0.816367	0.0010
25	2.984728	0.8065	2.832526	0.816367	0.0010
26	2.984727	0.8065	2.832526	0.816367	0.0010
27	2.984728	0.8065	2.832526	0.816367	0.0010
28	2.984728	0.8065	2.832526	0.816367	0.0005
29	2.984727	0.8065	2.832526	0.816367	0.0005
30	2.984728	0.8065	2.832526	0.816367	0.0005
31	2.984727	0.8065	2.832526	0.816367	0.0005
32	2.984727	0.8065	2.832526	0.816367	0.0005

Navie

query= Input(name='text_left', shape=(max_len,))
doc  = Input(name='text_right', shape=(max_len,))
x=concatenate([query,doc])
x_out = Dense(1, activation='linear')(x)
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model=Model(inputs=[query, doc],outputs=x_out,name="Navie")
model.summary()

Model: "Navie"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
concatenate_1 (Concatenate)     (None, 30)           0           text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
dense_3 (Dense)                 (None, 1)            31          concatenate_1[0][0]              
==================================================================================================
Total params: 31
Trainable params: 31
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 9ms/step - loss: 8.9142 - accuracy: 0.4155 - val_loss: 8.2654 - val_accuracy: 0.4571

Epoch 00001: val_accuracy improved from -inf to 0.45709, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 5ms/step - loss: 8.0908 - accuracy: 0.4679 - val_loss: 7.8712 - val_accuracy: 0.4830

Epoch 00002: val_accuracy improved from 0.45709 to 0.48303, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 3/50
63/63 [==============================] - 0s 4ms/step - loss: 7.5510 - accuracy: 0.5045 - val_loss: 7.6945 - val_accuracy: 0.4950

Epoch 00003: val_accuracy improved from 0.48303 to 0.49501, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 4/50
63/63 [==============================] - 0s 4ms/step - loss: 7.8960 - accuracy: 0.4823 - val_loss: 7.4436 - val_accuracy: 0.5130

Epoch 00004: val_accuracy improved from 0.49501 to 0.51297, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 5/50
63/63 [==============================] - 0s 4ms/step - loss: 7.1944 - accuracy: 0.5268 - val_loss: 7.1445 - val_accuracy: 0.5309

Epoch 00005: val_accuracy improved from 0.51297 to 0.53094, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 6/50
63/63 [==============================] - 0s 4ms/step - loss: 7.2356 - accuracy: 0.5242 - val_loss: 6.7534 - val_accuracy: 0.5549

Epoch 00006: val_accuracy improved from 0.53094 to 0.55489, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 7/50
63/63 [==============================] - 0s 4ms/step - loss: 6.7651 - accuracy: 0.5518 - val_loss: 6.5213 - val_accuracy: 0.5669

Epoch 00007: val_accuracy improved from 0.55489 to 0.56687, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 8/50
63/63 [==============================] - 0s 4ms/step - loss: 6.2155 - accuracy: 0.5932 - val_loss: 6.8991 - val_accuracy: 0.5489

Epoch 00008: val_accuracy did not improve from 0.56687
Epoch 9/50
63/63 [==============================] - 0s 3ms/step - loss: 6.4371 - accuracy: 0.5759 - val_loss: 6.8685 - val_accuracy: 0.5509

Epoch 00009: val_accuracy did not improve from 0.56687
Epoch 10/50
63/63 [==============================] - 0s 3ms/step - loss: 6.5154 - accuracy: 0.5739 - val_loss: 6.4209 - val_accuracy: 0.5788

Epoch 00010: val_accuracy improved from 0.56687 to 0.57884, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 11/50
63/63 [==============================] - 0s 4ms/step - loss: 5.8972 - accuracy: 0.6133 - val_loss: 5.5397 - val_accuracy: 0.6327

Epoch 00011: val_accuracy improved from 0.57884 to 0.63273, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 12/50
63/63 [==============================] - 0s 4ms/step - loss: 5.7758 - accuracy: 0.6185 - val_loss: 5.2374 - val_accuracy: 0.6567

Epoch 00012: val_accuracy improved from 0.63273 to 0.65669, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 13/50
63/63 [==============================] - 0s 4ms/step - loss: 5.1468 - accuracy: 0.6625 - val_loss: 5.3260 - val_accuracy: 0.6507

Epoch 00013: val_accuracy did not improve from 0.65669
Epoch 14/50
63/63 [==============================] - 0s 4ms/step - loss: 5.1256 - accuracy: 0.6654 - val_loss: 5.3839 - val_accuracy: 0.6467

Epoch 00014: val_accuracy did not improve from 0.65669
Epoch 15/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1535 - accuracy: 0.6591 - val_loss: 5.0819 - val_accuracy: 0.6667

Epoch 00015: val_accuracy improved from 0.65669 to 0.66667, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 16/50
63/63 [==============================] - 0s 4ms/step - loss: 4.8616 - accuracy: 0.6807 - val_loss: 5.0814 - val_accuracy: 0.6647

Epoch 00016: val_accuracy did not improve from 0.66667
Epoch 17/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1588 - accuracy: 0.6602 - val_loss: 4.8977 - val_accuracy: 0.6786

Epoch 00017: val_accuracy improved from 0.66667 to 0.67864, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 18/50
63/63 [==============================] - 0s 4ms/step - loss: 4.8387 - accuracy: 0.6825 - val_loss: 4.8953 - val_accuracy: 0.6806

Epoch 00018: val_accuracy improved from 0.67864 to 0.68064, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 19/50
63/63 [==============================] - 0s 3ms/step - loss: 4.8239 - accuracy: 0.6855 - val_loss: 4.8953 - val_accuracy: 0.6806

Epoch 00019: val_accuracy did not improve from 0.68064
Epoch 20/50
63/63 [==============================] - 0s 3ms/step - loss: 4.9532 - accuracy: 0.6770 - val_loss: 4.8953 - val_accuracy: 0.6806

Epoch 00020: val_accuracy did not improve from 0.68064
Epoch 21/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1150 - accuracy: 0.6663 - val_loss: 4.8967 - val_accuracy: 0.6806

Epoch 00021: val_accuracy did not improve from 0.68064
Epoch 22/50
63/63 [==============================] - 0s 3ms/step - loss: 4.9409 - accuracy: 0.6763 - val_loss: 4.8350 - val_accuracy: 0.6846

Epoch 00022: val_accuracy improved from 0.68064 to 0.68463, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 23/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4491 - accuracy: 0.7095 - val_loss: 4.5635 - val_accuracy: 0.7006

Epoch 00023: val_accuracy improved from 0.68463 to 0.70060, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 24/50
63/63 [==============================] - 0s 4ms/step - loss: 4.9084 - accuracy: 0.6793 - val_loss: 4.5003 - val_accuracy: 0.7066

Epoch 00024: val_accuracy improved from 0.70060 to 0.70659, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 25/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5295 - accuracy: 0.7006 - val_loss: 4.5016 - val_accuracy: 0.7066

Epoch 00025: val_accuracy did not improve from 0.70659
Epoch 26/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4010 - accuracy: 0.7126 - val_loss: 4.5308 - val_accuracy: 0.7046

Epoch 00026: val_accuracy did not improve from 0.70659
Epoch 27/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5775 - accuracy: 0.7008 - val_loss: 4.4738 - val_accuracy: 0.7066

Epoch 00027: val_accuracy did not improve from 0.70659
Epoch 28/50
63/63 [==============================] - 0s 3ms/step - loss: 4.4680 - accuracy: 0.7084 - val_loss: 4.4743 - val_accuracy: 0.7046

Epoch 00028: val_accuracy did not improve from 0.70659
Epoch 29/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5811 - accuracy: 0.7003 - val_loss: 4.5315 - val_accuracy: 0.7046

Epoch 00029: val_accuracy did not improve from 0.70659
Epoch 30/50
63/63 [==============================] - 0s 4ms/step - loss: 4.7530 - accuracy: 0.6898 - val_loss: 4.5332 - val_accuracy: 0.7026

Epoch 00030: val_accuracy did not improve from 0.70659
Epoch 31/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5055 - accuracy: 0.7062 - val_loss: 4.5337 - val_accuracy: 0.7026

Epoch 00031: val_accuracy did not improve from 0.70659
Epoch 32/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4446 - accuracy: 0.7104 - val_loss: 4.5347 - val_accuracy: 0.7026

Epoch 00032: val_accuracy did not improve from 0.70659
Epoch 33/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5923 - accuracy: 0.7006 - val_loss: 5.2948 - val_accuracy: 0.6527

Epoch 00033: val_accuracy did not improve from 0.70659
Epoch 34/50
63/63 [==============================] - 0s 3ms/step - loss: 5.6213 - accuracy: 0.6318 - val_loss: 5.7545 - val_accuracy: 0.6188

Epoch 00034: val_accuracy did not improve from 0.70659
Epoch 35/50
63/63 [==============================] - 0s 3ms/step - loss: 5.7049 - accuracy: 0.6244 - val_loss: 5.4230 - val_accuracy: 0.6427

Epoch 00035: val_accuracy did not improve from 0.70659
Epoch 36/50
63/63 [==============================] - 0s 5ms/step - loss: 5.1246 - accuracy: 0.6643 - val_loss: 5.4163 - val_accuracy: 0.6427

Epoch 00036: val_accuracy did not improve from 0.70659
Epoch 37/50
63/63 [==============================] - 0s 3ms/step - loss: 5.1822 - accuracy: 0.6575 - val_loss: 4.9631 - val_accuracy: 0.6727

Epoch 00037: val_accuracy did not improve from 0.70659
Epoch 38/50
63/63 [==============================] - 0s 4ms/step - loss: 4.5541 - accuracy: 0.7030 - val_loss: 4.9609 - val_accuracy: 0.6727

Epoch 00038: val_accuracy did not improve from 0.70659
Epoch 39/50
63/63 [==============================] - 0s 3ms/step - loss: 4.5174 - accuracy: 0.7054 - val_loss: 4.9610 - val_accuracy: 0.6727

Epoch 00039: val_accuracy did not improve from 0.70659
Epoch 40/50
63/63 [==============================] - 0s 3ms/step - loss: 4.7210 - accuracy: 0.6921 - val_loss: 4.9607 - val_accuracy: 0.6747

Epoch 00040: val_accuracy did not improve from 0.70659
Epoch 41/50
63/63 [==============================] - 0s 4ms/step - loss: 4.7356 - accuracy: 0.6909 - val_loss: 4.5617 - val_accuracy: 0.7026

Epoch 00041: val_accuracy did not improve from 0.70659
Epoch 42/50
63/63 [==============================] - 0s 3ms/step - loss: 4.2498 - accuracy: 0.7209 - val_loss: 4.4727 - val_accuracy: 0.7066

Epoch 00042: val_accuracy did not improve from 0.70659
Epoch 43/50
63/63 [==============================] - 0s 4ms/step - loss: 4.4859 - accuracy: 0.7052 - val_loss: 4.0455 - val_accuracy: 0.7365

Epoch 00043: val_accuracy improved from 0.70659 to 0.73653, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 44/50
63/63 [==============================] - 0s 3ms/step - loss: 4.0808 - accuracy: 0.7324 - val_loss: 3.9542 - val_accuracy: 0.7425

Epoch 00044: val_accuracy improved from 0.73653 to 0.74251, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 45/50
63/63 [==============================] - 0s 4ms/step - loss: 4.0209 - accuracy: 0.7360 - val_loss: 3.9542 - val_accuracy: 0.7425

Epoch 00045: val_accuracy did not improve from 0.74251
Epoch 46/50
63/63 [==============================] - 0s 4ms/step - loss: 4.1993 - accuracy: 0.7262 - val_loss: 3.9272 - val_accuracy: 0.7425

Epoch 00046: val_accuracy did not improve from 0.74251
Epoch 47/50
63/63 [==============================] - 0s 4ms/step - loss: 4.2362 - accuracy: 0.7233 - val_loss: 3.8685 - val_accuracy: 0.7465

Epoch 00047: val_accuracy improved from 0.74251 to 0.74651, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\weights.hdf5
Epoch 48/50
63/63 [==============================] - 0s 3ms/step - loss: 3.9913 - accuracy: 0.7392 - val_loss: 3.8669 - val_accuracy: 0.7465

Epoch 00048: val_accuracy did not improve from 0.74651
Epoch 49/50
63/63 [==============================] - 0s 3ms/step - loss: 4.1925 - accuracy: 0.7268 - val_loss: 3.8673 - val_accuracy: 0.7465

Epoch 00049: val_accuracy did not improve from 0.74651
Epoch 50/50
63/63 [==============================] - 0s 3ms/step - loss: 4.3917 - accuracy: 0.7131 - val_loss: 3.8678 - val_accuracy: 0.7465

Epoch 00050: val_accuracy did not improve from 0.74651
Navie运行时间: 0.27 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\save\Navie\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\Navie\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-rz5OIGiQ-1623898674921)(output_46_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-24kczFoC-1623898674923)(output_46_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	8.703827	0.4280	8.265403	0.457086	0.001
2	8.127553	0.4645	7.871212	0.483034	0.001
3	7.788067	0.4895	7.694490	0.495010	0.001
4	7.614698	0.5000	7.443610	0.512974	0.001
5	7.249546	0.5235	7.144458	0.530938	0.001
6	7.061227	0.5370	6.753396	0.554890	0.001
7	6.457839	0.5735	6.521289	0.566866	0.001
8	6.580623	0.5690	6.899066	0.548902	0.001
9	6.656266	0.5630	6.868452	0.550898	0.001
10	6.582748	0.5685	6.420915	0.578842	0.001
11	5.829680	0.6170	5.539738	0.632735	0.001
12	5.300040	0.6505	5.237395	0.656687	0.001
13	5.115226	0.6650	5.325986	0.650699	0.001
14	5.164371	0.6630	5.383919	0.646707	0.001
15	5.033154	0.6685	5.081899	0.666667	0.001
16	4.999567	0.6720	5.081350	0.664671	0.001
17	4.864917	0.6805	4.897707	0.678643	0.001
18	4.762197	0.6890	4.895268	0.680639	0.001
19	4.761465	0.6895	4.895268	0.680639	0.001
20	4.761386	0.6895	4.895268	0.680639	0.001
21	4.763627	0.6885	4.896749	0.680639	0.001
22	4.762402	0.6890	4.834998	0.684631	0.001
23	4.664893	0.6945	4.563492	0.700599	0.001
24	4.574459	0.7005	4.500281	0.706587	0.001
25	4.543370	0.7025	4.501622	0.706587	0.001
26	4.542712	0.7030	4.530774	0.704591	0.001
27	4.542055	0.7035	4.473816	0.706587	0.001
28	4.542201	0.7030	4.474332	0.704591	0.001
29	4.541991	0.7035	4.531502	0.704591	0.001
30	4.541663	0.7035	4.533184	0.702595	0.001
31	4.541536	0.7035	4.533740	0.702595	0.001
32	4.541585	0.7040	4.534698	0.702595	0.001
33	4.577245	0.7005	5.294760	0.652695	0.001
34	5.759947	0.6225	5.754533	0.618762	0.001
35	5.581280	0.6330	5.423037	0.642715	0.001
36	5.440868	0.6430	5.416341	0.642715	0.001
37	4.959578	0.6740	4.963100	0.672655	0.001
38	4.746535	0.6905	4.960928	0.672655	0.001
39	4.746130	0.6905	4.960978	0.672655	0.001
40	4.746026	0.6905	4.960693	0.674651	0.001
41	4.634393	0.6970	4.561750	0.702595	0.001
42	4.531757	0.7015	4.472662	0.706587	0.001
43	4.434167	0.7095	4.045545	0.736527	0.001
44	4.260184	0.7205	3.954157	0.742515	0.001
45	4.174057	0.7260	3.954157	0.742515	0.001
46	4.110280	0.7315	3.927210	0.742515	0.001
47	4.095524	0.7325	3.868510	0.746507	0.001
48	4.086236	0.7335	3.866899	0.746507	0.001
49	4.086092	0.7335	3.867256	0.746507	0.001
50	4.086073	0.7335	3.867835	0.746507	0.001

cdssm

def textCNN(x):
    x=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',
        kernel_initializer='glorot_normal',bias_initializer='zeros')(x)
    # Apply max pooling by take max at each dimension across
    # all word_trigram features.
    x =Dropout(0.3)(x)
    x = GlobalMaxPool1D()(x)
    for _ in range(3):
        x = Dense(128,activation='relu')(x)
    return x

filters = 200
kernel_sizes = [3,4,5]
output_dim = 100

input1 = keras.layers.Input(name='sent1', shape=(max_len,))
input2 = keras.layers.Input(name='sent2', shape=(max_len,))

embedding = keras.layers.Embedding(vocab_size, embedding_size)
sent1_embed = embedding(input1)
sent2_embed = embedding(input2)

# [output_sent1, output_sent2]=[textCNN(sent1_embed),textCNN(sent2_embed)]

xa = [textCNN(sent1_embed),textCNN(sent2_embed)]

# xa = Dot(axes=[1, 1], normalize=True)(xa)

cosine_output = keras.layers.Dot(axes=[1, 1], normalize=True)(xa)
outputs = keras.layers.Dense(1, activation='linear', name="output")(cosine_output)

model = keras.models.Model(inputs=[input1, input2], outputs=outputs,name="CDSSM")
model.summary()

Model: "CDSSM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
sent1 (InputLayer)              [(None, 15)]         0                                            
__________________________________________________________________________________________________
sent2 (InputLayer)              [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding (Embedding)           (None, 15, 128)      278400      sent1[0][0]                      
                                                                 sent2[0][0]                      
__________________________________________________________________________________________________
conv1d (Conv1D)                 (None, 15, 32)       12320       embedding[0][0]                  
__________________________________________________________________________________________________
conv1d_1 (Conv1D)               (None, 15, 32)       12320       embedding[1][0]                  
__________________________________________________________________________________________________
dropout (Dropout)               (None, 15, 32)       0           conv1d[0][0]                     
__________________________________________________________________________________________________
dropout_1 (Dropout)             (None, 15, 32)       0           conv1d_1[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d (GlobalMax (None, 32)           0           dropout[0][0]                    
__________________________________________________________________________________________________
global_max_pooling1d_1 (GlobalM (None, 32)           0           dropout_1[0][0]                  
__________________________________________________________________________________________________
dense_4 (Dense)                 (None, 128)          4224        global_max_pooling1d[0][0]       
__________________________________________________________________________________________________
dense_7 (Dense)                 (None, 128)          4224        global_max_pooling1d_1[0][0]     
__________________________________________________________________________________________________
dense_5 (Dense)                 (None, 128)          16512       dense_4[0][0]                    
__________________________________________________________________________________________________
dense_8 (Dense)                 (None, 128)          16512       dense_7[0][0]                    
__________________________________________________________________________________________________
dense_6 (Dense)                 (None, 128)          16512       dense_5[0][0]                    
__________________________________________________________________________________________________
dense_9 (Dense)                 (None, 128)          16512       dense_8[0][0]                    
__________________________________________________________________________________________________
dot (Dot)                       (None, 1)            0           dense_6[0][0]                    
                                                                 dense_9[0][0]                    
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           dot[0][0]                        
==================================================================================================
Total params: 377,538
Trainable params: 377,538
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 5s 20ms/step - loss: 0.5117 - accuracy: 0.8011 - val_loss: 0.4879 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 11ms/step - loss: 0.5048 - accuracy: 0.7961 - val_loss: 0.4895 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 0.4382 - accuracy: 0.8074 - val_loss: 0.4815 - val_accuracy: 0.7984

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 11ms/step - loss: 0.3467 - accuracy: 0.8669 - val_loss: 0.5471 - val_accuracy: 0.7465

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 11ms/step - loss: 0.3072 - accuracy: 0.8672 - val_loss: 0.5465 - val_accuracy: 0.7725

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2380 - accuracy: 0.9023 - val_loss: 0.6638 - val_accuracy: 0.6766

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2054 - accuracy: 0.9284 - val_loss: 0.6456 - val_accuracy: 0.7565

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1676 - accuracy: 0.9392 - val_loss: 0.7225 - val_accuracy: 0.7246

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1127 - accuracy: 0.9616 - val_loss: 1.2580 - val_accuracy: 0.7545

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1236 - accuracy: 0.9582 - val_loss: 1.9337 - val_accuracy: 0.7924

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1590 - accuracy: 0.9407 - val_loss: 1.5420 - val_accuracy: 0.7505

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0837 - accuracy: 0.9729 - val_loss: 2.0833 - val_accuracy: 0.7665

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1183 - accuracy: 0.9739 - val_loss: 2.1034 - val_accuracy: 0.7246

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1002 - accuracy: 0.9581 - val_loss: 2.0305 - val_accuracy: 0.7086

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0706 - accuracy: 0.9759 - val_loss: 1.8748 - val_accuracy: 0.7485

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0770 - accuracy: 0.9747 - val_loss: 1.6595 - val_accuracy: 0.7924

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1662 - accuracy: 0.9520 - val_loss: 1.6988 - val_accuracy: 0.7605

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1409 - accuracy: 0.9498 - val_loss: 0.9682 - val_accuracy: 0.7645

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0659 - accuracy: 0.9723 - val_loss: 1.7093 - val_accuracy: 0.7285

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1253 - accuracy: 0.9629 - val_loss: 0.4928 - val_accuracy: 0.8144

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 12ms/step - loss: 0.3694 - accuracy: 0.8108 - val_loss: 1.0434 - val_accuracy: 0.7745

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 11ms/step - loss: 0.2163 - accuracy: 0.9157 - val_loss: 1.3512 - val_accuracy: 0.7605

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1188 - accuracy: 0.9498 - val_loss: 1.5283 - val_accuracy: 0.7764

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 11ms/step - loss: 0.1007 - accuracy: 0.9636 - val_loss: 1.9045 - val_accuracy: 0.7545

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0744 - accuracy: 0.9786 - val_loss: 1.3289 - val_accuracy: 0.7585

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0813 - accuracy: 0.9758 - val_loss: 1.8299 - val_accuracy: 0.7525

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0597 - accuracy: 0.9770 - val_loss: 2.0222 - val_accuracy: 0.7405

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0912 - accuracy: 0.9732 - val_loss: 1.7747 - val_accuracy: 0.7605

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0732 - accuracy: 0.9780 - val_loss: 2.1430 - val_accuracy: 0.7585

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0303 - accuracy: 0.9893 - val_loss: 2.0570 - val_accuracy: 0.7365

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0335 - accuracy: 0.9858 - val_loss: 2.2736 - val_accuracy: 0.7525

Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0299 - accuracy: 0.9884 - val_loss: 2.1349 - val_accuracy: 0.7525

Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0418 - accuracy: 0.9923 - val_loss: 1.9088 - val_accuracy: 0.7585

Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0319 - accuracy: 0.9870 - val_loss: 2.1658 - val_accuracy: 0.7405

Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0511 - accuracy: 0.9873 - val_loss: 2.1202 - val_accuracy: 0.7505

Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0156 - accuracy: 0.9948 - val_loss: 2.2616 - val_accuracy: 0.7605

Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0366 - accuracy: 0.9896 - val_loss: 2.1474 - val_accuracy: 0.7625

Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0294 - accuracy: 0.9939 - val_loss: 2.3427 - val_accuracy: 0.7545

Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0230 - accuracy: 0.9937 - val_loss: 2.4009 - val_accuracy: 0.7585

Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 10ms/step - loss: 0.0484 - accuracy: 0.9920 - val_loss: 2.4710 - val_accuracy: 0.7685

Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0363 - accuracy: 0.9868 - val_loss: 2.2250 - val_accuracy: 0.7465

Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0194 - accuracy: 0.9907 - val_loss: 2.4387 - val_accuracy: 0.7625

Epoch 00042: val_accuracy did not improve from 0.81437
Epoch 43/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0297 - accuracy: 0.9878 - val_loss: 2.3372 - val_accuracy: 0.7505

Epoch 00043: val_accuracy did not improve from 0.81437
Epoch 44/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0251 - accuracy: 0.9923 - val_loss: 2.4880 - val_accuracy: 0.7425

Epoch 00044: val_accuracy did not improve from 0.81437
Epoch 45/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0354 - accuracy: 0.9895 - val_loss: 2.1338 - val_accuracy: 0.7645

Epoch 00045: val_accuracy did not improve from 0.81437
Epoch 46/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0263 - accuracy: 0.9880 - val_loss: 2.3131 - val_accuracy: 0.7246

Epoch 00046: val_accuracy did not improve from 0.81437
Epoch 47/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0194 - accuracy: 0.9941 - val_loss: 2.3929 - val_accuracy: 0.7365

Epoch 00047: val_accuracy did not improve from 0.81437
Epoch 48/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0197 - accuracy: 0.9922 - val_loss: 2.3312 - val_accuracy: 0.7305

Epoch 00048: val_accuracy did not improve from 0.81437
Epoch 49/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0225 - accuracy: 0.9883 - val_loss: 2.3285 - val_accuracy: 0.7525

Epoch 00049: val_accuracy did not improve from 0.81437
Epoch 50/50
63/63 [==============================] - 1s 11ms/step - loss: 0.0246 - accuracy: 0.9921 - val_loss: 2.2293 - val_accuracy: 0.7325

Epoch 00050: val_accuracy did not improve from 0.81437
CDSSM运行时间: 0.76 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\save\CDSSM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\CDSSM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jAlOz8Da-1623898674923)(output_52_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-ZXVcTZna-1623898674924)(output_52_1.svg)]

os.path.join(path,'save',model.name,model.name+'.csv')

'H:\\code\\BNLP学习\\NLP项目教程\\匹配\\文本匹配\\花呗问答匹配\\outputs\\model\\CDSSM\\save\\CDSSM\\CDSSM.csv'

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.502438	0.8060	0.487863	0.814371	0.001
2	0.482865	0.8060	0.489541	0.814371	0.001
3	0.443680	0.8125	0.481475	0.798403	0.001
4	0.362532	0.8495	0.547100	0.746507	0.001
5	0.315729	0.8590	0.546462	0.772455	0.001
6	0.249621	0.8995	0.663817	0.676647	0.001
7	0.210387	0.9230	0.645588	0.756487	0.001
8	0.177454	0.9345	0.722511	0.724551	0.001
9	0.140597	0.9510	1.257960	0.754491	0.001
10	0.109115	0.9615	1.933702	0.792415	0.001
11	0.146496	0.9490	1.541986	0.750499	0.001
12	0.104541	0.9680	2.083340	0.766467	0.001
13	0.117658	0.9735	2.103375	0.724551	0.001
14	0.106086	0.9615	2.030532	0.708583	0.001
15	0.072325	0.9735	1.874843	0.748503	0.001
16	0.130131	0.9575	1.659451	0.792415	0.001
17	0.147002	0.9540	1.698843	0.760479	0.001
18	0.140070	0.9525	0.968186	0.764471	0.001
19	0.070261	0.9710	1.709319	0.728543	0.001
20	0.259516	0.9030	0.492770	0.814371	0.001
21	0.316189	0.8490	1.043425	0.774451	0.001
22	0.204753	0.9255	1.351184	0.760479	0.001
23	0.127438	0.9505	1.528280	0.776447	0.001
24	0.088141	0.9675	1.904538	0.754491	0.001
25	0.093163	0.9700	1.328880	0.758483	0.001
26	0.070672	0.9770	1.829938	0.752495	0.001
27	0.058763	0.9765	2.022192	0.740519	0.001
28	0.104254	0.9665	1.774662	0.760479	0.001
29	0.056904	0.9785	2.143026	0.758483	0.001
30	0.040742	0.9860	2.056966	0.736527	0.001
31	0.032480	0.9875	2.273608	0.752495	0.001
32	0.033481	0.9880	2.134907	0.752495	0.001
33	0.042117	0.9890	1.908836	0.758483	0.001
34	0.027738	0.9880	2.165784	0.740519	0.001
35	0.045009	0.9875	2.120193	0.750499	0.001
36	0.020214	0.9935	2.261635	0.760479	0.001
37	0.040293	0.9890	2.147407	0.762475	0.001
38	0.027351	0.9925	2.342712	0.754491	0.001
39	0.027095	0.9910	2.400923	0.758483	0.001
40	0.037127	0.9920	2.471040	0.768463	0.001
41	0.038542	0.9855	2.225006	0.746507	0.001
42	0.022141	0.9900	2.438666	0.762475	0.001
43	0.034390	0.9870	2.337209	0.750499	0.001
44	0.017784	0.9955	2.488035	0.742515	0.001
45	0.032875	0.9920	2.133770	0.764471	0.001
46	0.022209	0.9915	2.313134	0.724551	0.001
47	0.024295	0.9935	2.392854	0.736527	0.001
48	0.016932	0.9940	2.331223	0.730539	0.001
49	0.025668	0.9915	2.328529	0.752495	0.001
50	0.023949	0.9920	2.229317	0.732535	0.001

BasictextCNN

def BaseTextCNN(inputs):
    conv_layers = []
    max_poolings = []
    for kernel_size in kernel_sizes:
        conv_layers.append(
            keras.layers.Conv1D(filters=filters, kernel_size=kernel_size, 
                            activation='relu', padding="same")
        )
        max_poolings.append(keras.layers.GlobalMaxPool1D())
    concatenate = keras.layers.Concatenate()
    dense = keras.layers.Dense(output_dim, activation='tanh')
    
    convs = []
    for i in range(len(kernel_sizes)):
        x = conv_layers[i](inputs)
        x = max_poolings[i](x)
        convs.append(x)
    x = concatenate(convs)
    output = dense(x)
    
    return output

filters = 200
kernel_sizes = [3,4,5]
output_dim = 100

input1 = keras.layers.Input(name='sent1', shape=(max_len,))
input2 = keras.layers.Input(name='sent2', shape=(max_len,))

embedding = keras.layers.Embedding(vocab_size, embedding_size)
sent1_embed = embedding(input1)
sent2_embed = embedding(input2)

output_sent1 = BaseTextCNN(sent1_embed)
output_sent2 = BaseTextCNN(sent2_embed)

cosine_output = keras.layers.Dot(axes=[1, 1], normalize=True)([output_sent1, output_sent2])
outputs = keras.layers.Dense(1, activation='linear', name="output")(cosine_output)

model = keras.models.Model(inputs=[input1, input2], outputs=outputs,name="BaseTextCNN_match")
model.summary()

Model: "BaseTextCNN_match"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
sent1 (InputLayer)              [(None, 15)]         0                                            
__________________________________________________________________________________________________
sent2 (InputLayer)              [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_1 (Embedding)         (None, 15, 128)      278400      sent1[0][0]                      
                                                                 sent2[0][0]                      
__________________________________________________________________________________________________
conv1d_2 (Conv1D)               (None, 15, 200)      77000       embedding_1[0][0]                
__________________________________________________________________________________________________
conv1d_3 (Conv1D)               (None, 15, 200)      102600      embedding_1[0][0]                
__________________________________________________________________________________________________
conv1d_4 (Conv1D)               (None, 15, 200)      128200      embedding_1[0][0]                
__________________________________________________________________________________________________
conv1d_5 (Conv1D)               (None, 15, 200)      77000       embedding_1[1][0]                
__________________________________________________________________________________________________
conv1d_6 (Conv1D)               (None, 15, 200)      102600      embedding_1[1][0]                
__________________________________________________________________________________________________
conv1d_7 (Conv1D)               (None, 15, 200)      128200      embedding_1[1][0]                
__________________________________________________________________________________________________
global_max_pooling1d_2 (GlobalM (None, 200)          0           conv1d_2[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_3 (GlobalM (None, 200)          0           conv1d_3[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_4 (GlobalM (None, 200)          0           conv1d_4[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_5 (GlobalM (None, 200)          0           conv1d_5[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_6 (GlobalM (None, 200)          0           conv1d_6[0][0]                   
__________________________________________________________________________________________________
global_max_pooling1d_7 (GlobalM (None, 200)          0           conv1d_7[0][0]                   
__________________________________________________________________________________________________
concatenate_2 (Concatenate)     (None, 600)          0           global_max_pooling1d_2[0][0]     
                                                                 global_max_pooling1d_3[0][0]     
                                                                 global_max_pooling1d_4[0][0]     
__________________________________________________________________________________________________
concatenate_3 (Concatenate)     (None, 600)          0           global_max_pooling1d_5[0][0]     
                                                                 global_max_pooling1d_6[0][0]     
                                                                 global_max_pooling1d_7[0][0]     
__________________________________________________________________________________________________
dense_10 (Dense)                (None, 100)          60100       concatenate_2[0][0]              
__________________________________________________________________________________________________
dense_11 (Dense)                (None, 100)          60100       concatenate_3[0][0]              
__________________________________________________________________________________________________
dot_1 (Dot)                     (None, 1)            0           dense_10[0][0]                   
                                                                 dense_11[0][0]                   
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           dot_1[0][0]                      
==================================================================================================
Total params: 1,014,202
Trainable params: 1,014,202
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 2s 24ms/step - loss: 0.5421 - accuracy: 0.7956 - val_loss: 0.4770 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 13ms/step - loss: 0.4627 - accuracy: 0.8030 - val_loss: 0.4782 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 0.4340 - accuracy: 0.7894 - val_loss: 0.8211 - val_accuracy: 0.8144

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 13ms/step - loss: 0.3410 - accuracy: 0.8213 - val_loss: 0.7917 - val_accuracy: 0.7964

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 13ms/step - loss: 0.2444 - accuracy: 0.8669 - val_loss: 0.8264 - val_accuracy: 0.7385

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1985 - accuracy: 0.9501 - val_loss: 1.7119 - val_accuracy: 0.7944

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1374 - accuracy: 0.9921 - val_loss: 1.6702 - val_accuracy: 0.7844

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1125 - accuracy: 0.9960 - val_loss: 1.7951 - val_accuracy: 0.7784

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 13ms/step - loss: 0.2621 - accuracy: 0.8833 - val_loss: 1.2079 - val_accuracy: 0.7864

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 13ms/step - loss: 0.1658 - accuracy: 0.9579 - val_loss: 1.4842 - val_accuracy: 0.7305

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0570 - accuracy: 0.9964 - val_loss: 1.7038 - val_accuracy: 0.7685

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0102 - accuracy: 0.9995 - val_loss: 1.8691 - val_accuracy: 0.7525

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0146 - accuracy: 0.9991 - val_loss: 1.9849 - val_accuracy: 0.7545

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0209 - accuracy: 0.9987 - val_loss: 1.9464 - val_accuracy: 0.7545

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0297 - accuracy: 0.9981 - val_loss: 1.9654 - val_accuracy: 0.7565

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0332 - accuracy: 0.9979 - val_loss: 1.9614 - val_accuracy: 0.7605

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0386 - accuracy: 0.9975 - val_loss: 1.9540 - val_accuracy: 0.7625

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0446 - accuracy: 0.9971 - val_loss: 1.9512 - val_accuracy: 0.7625

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - ETA: 0s - loss: 0.0101 - accuracy: 0.99 - 1s 13ms/step - loss: 0.0108 - accuracy: 0.9993 - val_loss: 1.9319 - val_accuracy: 0.7605

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0216 - accuracy: 0.9986 - val_loss: 1.9300 - val_accuracy: 0.7625

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0397 - accuracy: 0.9974 - val_loss: 1.8891 - val_accuracy: 0.7585

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0320 - accuracy: 0.9979 - val_loss: 1.9140 - val_accuracy: 0.7645

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0119 - accuracy: 0.9992 - val_loss: 1.9261 - val_accuracy: 0.7545

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0292 - accuracy: 0.9981 - val_loss: 1.9557 - val_accuracy: 0.7445

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0224 - accuracy: 0.9986 - val_loss: 2.0265 - val_accuracy: 0.7465

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0156 - accuracy: 0.9990 - val_loss: 2.0036 - val_accuracy: 0.7505

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0550 - accuracy: 0.9964 - val_loss: 1.9990 - val_accuracy: 0.7445

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0418 - accuracy: 0.9973 - val_loss: 2.1372 - val_accuracy: 0.7585

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0207 - accuracy: 0.9987 - val_loss: 2.0270 - val_accuracy: 0.7525

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0339 - accuracy: 0.9978 - val_loss: 2.0258 - val_accuracy: 0.7545

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0199 - accuracy: 0.9987 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0322 - accuracy: 0.9979 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0416 - accuracy: 0.9973 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0351 - accuracy: 0.9977 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0467 - accuracy: 0.9970 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0197 - accuracy: 0.9987 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0113 - accuracy: 0.9993 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0317 - accuracy: 0.9979 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0348 - accuracy: 0.9977 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0219 - accuracy: 0.9986 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 13ms/step - loss: 0.0225 - accuracy: 0.9985 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0189 - accuracy: 0.9988 - val_loss: 1.9759 - val_accuracy: 0.7485

Epoch 00042: val_accuracy did not improve from 0.81437
BaseTextCNN_match运行时间: 0.72 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\save\BaseTextCNN_match\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\BaseTextCNN_match\save

DSSM

base_network = _make_multi_layer_perceptron_layer()

input_left=Input(name='text_left', shape=(max_len,))
input_right = Input(name='text_right', shape=(max_len,))
# embedding = Embedding(vocab_size, embedding_size)
# embed_left = embedding(input_left)
# embed_right = embedding(input_right)
x = [base_network(input_left),base_network(input_right)]

x = Dot(axes=[1, 1], normalize=True)(x)
    
# Make output layer
x_out =  Dense(1, activation='linear')(x)

# self._backend =

# out =
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model= keras.Model(inputs=[input_left, input_right], outputs=x_out,name="DSSM")
model.summary()

Model: "DSSM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
dense_12 (Dense)                (None, 128)          2048        text_left[0][0]                  
__________________________________________________________________________________________________
dense_16 (Dense)                (None, 128)          2048        text_right[0][0]                 
__________________________________________________________________________________________________
dense_13 (Dense)                (None, 128)          16512       dense_12[0][0]                   
__________________________________________________________________________________________________
dense_17 (Dense)                (None, 128)          16512       dense_16[0][0]                   
__________________________________________________________________________________________________
dense_14 (Dense)                (None, 128)          16512       dense_13[0][0]                   
__________________________________________________________________________________________________
dense_18 (Dense)                (None, 128)          16512       dense_17[0][0]                   
__________________________________________________________________________________________________
dense_15 (Dense)                (None, 64)           8256        dense_14[0][0]                   
__________________________________________________________________________________________________
dense_19 (Dense)                (None, 64)           8256        dense_18[0][0]                   
__________________________________________________________________________________________________
dot_2 (Dot)                     (None, 1)            0           dense_15[0][0]                   
                                                                 dense_19[0][0]                   
__________________________________________________________________________________________________
dense_20 (Dense)                (None, 1)            2           dot_2[0][0]                      
==================================================================================================
Total params: 86,658
Trainable params: 86,658
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 13ms/step - loss: 2.9754 - accuracy: 0.8071 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8511 - accuracy: 0.8152 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9824 - accuracy: 0.8066 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9774 - accuracy: 0.8070 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0168 - accuracy: 0.8044 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0824 - accuracy: 0.8002 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0305 - accuracy: 0.8035 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1376 - accuracy: 0.7966 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1666 - accuracy: 0.7947 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 0s 6ms/step - loss: 2.7342 - accuracy: 0.8227 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9489 - accuracy: 0.8088 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0766 - accuracy: 0.8005 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 0s 7ms/step - loss: 2.9885 - accuracy: 0.8063 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9523 - accuracy: 0.8086 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0373 - accuracy: 0.8031 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9053 - accuracy: 0.8117 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1339 - accuracy: 0.7968 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 0s 6ms/step - loss: 3.2538 - accuracy: 0.7891 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0011 - accuracy: 0.8054 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8825 - accuracy: 0.8131 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1495 - accuracy: 0.7958 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9505 - accuracy: 0.8087 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1017 - accuracy: 0.7989 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9187 - accuracy: 0.8108 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9029 - accuracy: 0.8118 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 0s 6ms/step - loss: 2.8358 - accuracy: 0.8162 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1718 - accuracy: 0.7944 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 0s 6ms/step - loss: 3.0214 - accuracy: 0.8041 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 0s 6ms/step - loss: 3.1342 - accuracy: 0.7968 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9440 - accuracy: 0.8091 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 0s 6ms/step - loss: 2.9426 - accuracy: 0.8092 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00031: val_accuracy did not improve from 0.81437
DSSM运行时间: 0.26 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\save\DSSM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DSSM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-8LunGROM-1623898674925)(output_62_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-gXawIqmy-1623898674925)(output_62_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	2.992440	0.806	2.863314	0.814371	0.0010
2	2.992440	0.806	2.863314	0.814371	0.0010
3	2.992440	0.806	2.863314	0.814371	0.0010
4	2.992440	0.806	2.863314	0.814371	0.0010
5	2.992440	0.806	2.863314	0.814371	0.0010
6	2.992440	0.806	2.863314	0.814371	0.0010
7	2.992440	0.806	2.863314	0.814371	0.0010
8	2.992439	0.806	2.863314	0.814371	0.0010
9	2.992440	0.806	2.863314	0.814371	0.0010
10	2.992440	0.806	2.863314	0.814371	0.0010
11	2.992440	0.806	2.863314	0.814371	0.0010
12	2.992440	0.806	2.863314	0.814371	0.0010
13	2.992440	0.806	2.863314	0.814371	0.0010
14	2.992440	0.806	2.863314	0.814371	0.0010
15	2.992440	0.806	2.863314	0.814371	0.0010
16	2.992439	0.806	2.863314	0.814371	0.0010
17	2.992440	0.806	2.863314	0.814371	0.0010
18	2.992439	0.806	2.863314	0.814371	0.0010
19	2.992440	0.806	2.863314	0.814371	0.0010
20	2.992441	0.806	2.863314	0.814371	0.0010
21	2.992440	0.806	2.863314	0.814371	0.0010
22	2.992440	0.806	2.863314	0.814371	0.0010
23	2.992440	0.806	2.863314	0.814371	0.0010
24	2.992441	0.806	2.863314	0.814371	0.0010
25	2.992440	0.806	2.863314	0.814371	0.0010
26	2.992440	0.806	2.863314	0.814371	0.0010
27	2.992440	0.806	2.863314	0.814371	0.0005
28	2.992440	0.806	2.863314	0.814371	0.0005
29	2.992440	0.806	2.863314	0.814371	0.0005
30	2.992440	0.806	2.863314	0.814371	0.0005
31	2.992440	0.806	2.863314	0.814371	0.0005

ArcI

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
cnn1=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',kernel_initializer='glorot_normal',bias_initializer='zeros')(sent1_embed)
cnn2=Conv1D(filters=32,kernel_size=3,strides=1,padding='same',activation='relu',kernel_initializer='glorot_normal',bias_initializer='zeros')(sent2_embed)
cnn1=MaxPooling1D(pool_size=2)(cnn1)
flat1=Flatten()(cnn1)
cnn2=MaxPooling1D(pool_size=2)(cnn2)
flat2=Flatten()(cnn2)
# concatenate_1=concatenate([input1,input2],axis=-1)
concat =Concatenate(axis=1)([flat1, flat2])
dropout = Dropout(rate=0)(concat)
dense1=Dense(64)(dropout)
dense2=Dense(32)(dense1)
# dense3=Dense(128)(dense2)
outputs=Dense(1,activation='sigmoid',name="output")(dense2)
model=Model(inputs=[input1,input2],outputs=outputs,name="ArcI")
model.summary()

Model: "ArcI"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_2 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
conv1d_8 (Conv1D)               (None, 15, 32)       12320       embedding_2[0][0]                
__________________________________________________________________________________________________
conv1d_9 (Conv1D)               (None, 15, 32)       12320       embedding_2[1][0]                
__________________________________________________________________________________________________
max_pooling1d (MaxPooling1D)    (None, 7, 32)        0           conv1d_8[0][0]                   
__________________________________________________________________________________________________
max_pooling1d_1 (MaxPooling1D)  (None, 7, 32)        0           conv1d_9[0][0]                   
__________________________________________________________________________________________________
flatten (Flatten)               (None, 224)          0           max_pooling1d[0][0]              
__________________________________________________________________________________________________
flatten_1 (Flatten)             (None, 224)          0           max_pooling1d_1[0][0]            
__________________________________________________________________________________________________
concatenate_4 (Concatenate)     (None, 448)          0           flatten[0][0]                    
                                                                 flatten_1[0][0]                  
__________________________________________________________________________________________________
dropout_2 (Dropout)             (None, 448)          0           concatenate_4[0][0]              
__________________________________________________________________________________________________
dense_21 (Dense)                (None, 64)           28736       dropout_2[0][0]                  
__________________________________________________________________________________________________
dense_22 (Dense)                (None, 32)           2080        dense_21[0][0]                   
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            33          dense_22[0][0]                   
==================================================================================================
Total params: 333,889
Trainable params: 333,889
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 1s 16ms/step - loss: 0.5292 - accuracy: 0.7885 - val_loss: 0.4754 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 8ms/step - loss: 0.4580 - accuracy: 0.7951 - val_loss: 0.4823 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 8ms/step - loss: 0.3164 - accuracy: 0.8324 - val_loss: 0.6373 - val_accuracy: 0.7804

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 8ms/step - loss: 0.1099 - accuracy: 0.9617 - val_loss: 0.9353 - val_accuracy: 0.7745

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 9ms/step - loss: 0.0341 - accuracy: 0.9908 - val_loss: 1.1799 - val_accuracy: 0.7305

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 8ms/step - loss: 0.0083 - accuracy: 0.9990 - val_loss: 1.4070 - val_accuracy: 0.7565

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 9ms/step - loss: 0.0024 - accuracy: 1.0000 - val_loss: 1.5152 - val_accuracy: 0.7764

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 8ms/step - loss: 0.0011 - accuracy: 1.0000 - val_loss: 1.5363 - val_accuracy: 0.7465

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 8ms/step - loss: 5.5433e-04 - accuracy: 1.0000 - val_loss: 1.5963 - val_accuracy: 0.7505

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 8ms/step - loss: 4.5467e-04 - accuracy: 1.0000 - val_loss: 1.6363 - val_accuracy: 0.7525

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 8ms/step - loss: 3.1880e-04 - accuracy: 1.0000 - val_loss: 1.6832 - val_accuracy: 0.7565

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 8ms/step - loss: 2.2599e-04 - accuracy: 1.0000 - val_loss: 1.7060 - val_accuracy: 0.7545

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 9ms/step - loss: 2.0653e-04 - accuracy: 1.0000 - val_loss: 1.7369 - val_accuracy: 0.7545

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 8ms/step - loss: 1.7706e-04 - accuracy: 1.0000 - val_loss: 1.7675 - val_accuracy: 0.7545

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 8ms/step - loss: 1.4694e-04 - accuracy: 1.0000 - val_loss: 1.7911 - val_accuracy: 0.7525

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 8ms/step - loss: 1.3680e-04 - accuracy: 1.0000 - val_loss: 1.8178 - val_accuracy: 0.7525

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 9ms/step - loss: 1.1411e-04 - accuracy: 1.0000 - val_loss: 1.8362 - val_accuracy: 0.7525

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 9ms/step - loss: 1.0103e-04 - accuracy: 1.0000 - val_loss: 1.8574 - val_accuracy: 0.7505

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 10ms/step - loss: 8.8149e-05 - accuracy: 1.0000 - val_loss: 1.8767 - val_accuracy: 0.7505

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 8ms/step - loss: 8.0321e-05 - accuracy: 1.0000 - val_loss: 1.8971 - val_accuracy: 0.7505

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 8ms/step - loss: 6.7310e-05 - accuracy: 1.0000 - val_loss: 1.9162 - val_accuracy: 0.7505

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 8ms/step - loss: 6.2473e-05 - accuracy: 1.0000 - val_loss: 1.9336 - val_accuracy: 0.7505

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 9ms/step - loss: 5.3186e-05 - accuracy: 1.0000 - val_loss: 1.9502 - val_accuracy: 0.7505

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 9ms/step - loss: 4.9954e-05 - accuracy: 1.0000 - val_loss: 1.9679 - val_accuracy: 0.7505

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 8ms/step - loss: 4.3161e-05 - accuracy: 1.0000 - val_loss: 1.9824 - val_accuracy: 0.7505

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 8ms/step - loss: 4.5415e-05 - accuracy: 1.0000 - val_loss: 1.9985 - val_accuracy: 0.7505

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 9ms/step - loss: 3.9223e-05 - accuracy: 1.0000 - val_loss: 2.0130 - val_accuracy: 0.7505

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 8ms/step - loss: 3.7845e-05 - accuracy: 1.0000 - val_loss: 2.0262 - val_accuracy: 0.7505

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 9ms/step - loss: 3.3452e-05 - accuracy: 1.0000 - val_loss: 2.0399 - val_accuracy: 0.7485

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 8ms/step - loss: 3.2305e-05 - accuracy: 1.0000 - val_loss: 2.0535 - val_accuracy: 0.7485

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 8ms/step - loss: 3.0399e-05 - accuracy: 1.0000 - val_loss: 2.0668 - val_accuracy: 0.7505

Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 8ms/step - loss: 2.5758e-05 - accuracy: 1.0000 - val_loss: 2.0806 - val_accuracy: 0.7505

Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 8ms/step - loss: 2.3437e-05 - accuracy: 1.0000 - val_loss: 2.0864 - val_accuracy: 0.7525

Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 9ms/step - loss: 2.3289e-05 - accuracy: 1.0000 - val_loss: 2.0922 - val_accuracy: 0.7505

Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 9ms/step - loss: 2.3398e-05 - accuracy: 1.0000 - val_loss: 2.0987 - val_accuracy: 0.7505

Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 8ms/step - loss: 2.2757e-05 - accuracy: 1.0000 - val_loss: 2.1050 - val_accuracy: 0.7505

Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 9ms/step - loss: 2.0153e-05 - accuracy: 1.0000 - val_loss: 2.1116 - val_accuracy: 0.7525

Epoch 00037: val_accuracy did not improve from 0.81437
ArcI运行时间: 0.41 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\save\ArcI\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\ArcI\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-BcHVgoNZ-1623898674926)(output_67_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kg40iuAZ-1623898674926)(output_67_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.503494	0.8030	0.475359	0.814371	0.0010
2	0.438728	0.8060	0.482319	0.814371	0.0010
3	0.302796	0.8500	0.637273	0.780439	0.0010
4	0.116107	0.9580	0.935277	0.774451	0.0010
5	0.033786	0.9910	1.179889	0.730539	0.0010
6	0.008575	0.9985	1.406960	0.756487	0.0010
7	0.002379	1.0000	1.515246	0.776447	0.0010
8	0.000971	1.0000	1.536348	0.746507	0.0010
9	0.000542	1.0000	1.596271	0.750499	0.0010
10	0.000397	1.0000	1.636328	0.752495	0.0010
11	0.000304	1.0000	1.683157	0.756487	0.0010
12	0.000246	1.0000	1.706022	0.754491	0.0010
13	0.000203	1.0000	1.736940	0.754491	0.0010
14	0.000171	1.0000	1.767536	0.754491	0.0010
15	0.000146	1.0000	1.791100	0.752495	0.0010
16	0.000126	1.0000	1.817774	0.752495	0.0010
17	0.000110	1.0000	1.836237	0.752495	0.0010
18	0.000097	1.0000	1.857435	0.750499	0.0010
19	0.000086	1.0000	1.876659	0.750499	0.0010
20	0.000077	1.0000	1.897138	0.750499	0.0010
21	0.000069	1.0000	1.916157	0.750499	0.0010
22	0.000062	1.0000	1.933603	0.750499	0.0010
23	0.000056	1.0000	1.950206	0.750499	0.0010
24	0.000051	1.0000	1.967885	0.750499	0.0010
25	0.000047	1.0000	1.982365	0.750499	0.0010
26	0.000043	1.0000	1.998452	0.750499	0.0010
27	0.000039	1.0000	2.013018	0.750499	0.0010
28	0.000036	1.0000	2.026198	0.750499	0.0010
29	0.000033	1.0000	2.039930	0.748503	0.0010
30	0.000031	1.0000	2.053540	0.748503	0.0010
31	0.000029	1.0000	2.066777	0.750499	0.0010
32	0.000027	1.0000	2.080627	0.750499	0.0010
33	0.000025	1.0000	2.086367	0.752495	0.0005
34	0.000024	1.0000	2.092232	0.750499	0.0005
35	0.000023	1.0000	2.098683	0.750499	0.0005
36	0.000023	1.0000	2.104959	0.750499	0.0005
37	0.000022	1.0000	2.111558	0.752495	0.0005

KNRM

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
mm =Dot(axes=[2, 2], normalize=True)([sent1_embed, sent2_embed])
KM = []
for i in range(11):
    mu = 1. / (11 - 1) + (2. * i) / (11 - 1) - 1.0
    sigma = 0.1
    if mu > 1.0:
        sigma = 0.001
        mu = 1.0
    mm_exp = _kernel_layer(mu, sigma)(mm)
    mm_doc_sum = Lambda(
        lambda x: tf.reduce_sum(x, 2))(mm_exp)
    mm_log = Activation(tf.math.log1p)(mm_doc_sum)
    mm_sum = Lambda(
        lambda x: tf.reduce_sum(x, 1))(mm_log)
    KM.append(mm_sum)

phi = Lambda(lambda x: tf.stack(x, 1))(KM)
outputs=Dense(1,activation='sigmoid',name="output")(phi)
model=Model(inputs=[input1,input2],outputs=outputs,name="KNRM")
model.summary()

Model: "KNRM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_3 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
dot_3 (Dot)                     (None, 15, 15)       0           embedding_3[0][0]                
                                                                 embedding_3[1][0]                
__________________________________________________________________________________________________
activation (Activation)         (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_2 (Activation)       (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_4 (Activation)       (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_6 (Activation)       (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_8 (Activation)       (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_10 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_12 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_14 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_16 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_18 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
activation_20 (Activation)      (None, 15, 15)       0           dot_3[0][0]                      
__________________________________________________________________________________________________
lambda (Lambda)                 (None, 15)           0           activation[0][0]                 
__________________________________________________________________________________________________
lambda_2 (Lambda)               (None, 15)           0           activation_2[0][0]               
__________________________________________________________________________________________________
lambda_4 (Lambda)               (None, 15)           0           activation_4[0][0]               
__________________________________________________________________________________________________
lambda_6 (Lambda)               (None, 15)           0           activation_6[0][0]               
__________________________________________________________________________________________________
lambda_8 (Lambda)               (None, 15)           0           activation_8[0][0]               
__________________________________________________________________________________________________
lambda_10 (Lambda)              (None, 15)           0           activation_10[0][0]              
__________________________________________________________________________________________________
lambda_12 (Lambda)              (None, 15)           0           activation_12[0][0]              
__________________________________________________________________________________________________
lambda_14 (Lambda)              (None, 15)           0           activation_14[0][0]              
__________________________________________________________________________________________________
lambda_16 (Lambda)              (None, 15)           0           activation_16[0][0]              
__________________________________________________________________________________________________
lambda_18 (Lambda)              (None, 15)           0           activation_18[0][0]              
__________________________________________________________________________________________________
lambda_20 (Lambda)              (None, 15)           0           activation_20[0][0]              
__________________________________________________________________________________________________
activation_1 (Activation)       (None, 15)           0           lambda[0][0]                     
__________________________________________________________________________________________________
activation_3 (Activation)       (None, 15)           0           lambda_2[0][0]                   
__________________________________________________________________________________________________
activation_5 (Activation)       (None, 15)           0           lambda_4[0][0]                   
__________________________________________________________________________________________________
activation_7 (Activation)       (None, 15)           0           lambda_6[0][0]                   
__________________________________________________________________________________________________
activation_9 (Activation)       (None, 15)           0           lambda_8[0][0]                   
__________________________________________________________________________________________________
activation_11 (Activation)      (None, 15)           0           lambda_10[0][0]                  
__________________________________________________________________________________________________
activation_13 (Activation)      (None, 15)           0           lambda_12[0][0]                  
__________________________________________________________________________________________________
activation_15 (Activation)      (None, 15)           0           lambda_14[0][0]                  
__________________________________________________________________________________________________
activation_17 (Activation)      (None, 15)           0           lambda_16[0][0]                  
__________________________________________________________________________________________________
activation_19 (Activation)      (None, 15)           0           lambda_18[0][0]                  
__________________________________________________________________________________________________
activation_21 (Activation)      (None, 15)           0           lambda_20[0][0]                  
__________________________________________________________________________________________________
lambda_1 (Lambda)               (None,)              0           activation_1[0][0]               
__________________________________________________________________________________________________
lambda_3 (Lambda)               (None,)              0           activation_3[0][0]               
__________________________________________________________________________________________________
lambda_5 (Lambda)               (None,)              0           activation_5[0][0]               
__________________________________________________________________________________________________
lambda_7 (Lambda)               (None,)              0           activation_7[0][0]               
__________________________________________________________________________________________________
lambda_9 (Lambda)               (None,)              0           activation_9[0][0]               
__________________________________________________________________________________________________
lambda_11 (Lambda)              (None,)              0           activation_11[0][0]              
__________________________________________________________________________________________________
lambda_13 (Lambda)              (None,)              0           activation_13[0][0]              
__________________________________________________________________________________________________
lambda_15 (Lambda)              (None,)              0           activation_15[0][0]              
__________________________________________________________________________________________________
lambda_17 (Lambda)              (None,)              0           activation_17[0][0]              
__________________________________________________________________________________________________
lambda_19 (Lambda)              (None,)              0           activation_19[0][0]              
__________________________________________________________________________________________________
lambda_21 (Lambda)              (None,)              0           activation_21[0][0]              
__________________________________________________________________________________________________
lambda_22 (Lambda)              (None, 11)           0           lambda_1[0][0]                   
                                                                 lambda_3[0][0]                   
                                                                 lambda_5[0][0]                   
                                                                 lambda_7[0][0]                   
                                                                 lambda_9[0][0]                   
                                                                 lambda_11[0][0]                  
                                                                 lambda_13[0][0]                  
                                                                 lambda_15[0][0]                  
                                                                 lambda_17[0][0]                  
                                                                 lambda_19[0][0]                  
                                                                 lambda_21[0][0]                  
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            12          lambda_22[0][0]                  
==================================================================================================
Total params: 278,412
Trainable params: 278,412
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 3s 25ms/step - loss: 0.6209 - accuracy: 0.7793 - val_loss: 0.4904 - val_accuracy: 0.7944

Epoch 00001: val_accuracy improved from -inf to 0.79441, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 15ms/step - loss: 0.4332 - accuracy: 0.8002 - val_loss: 0.4921 - val_accuracy: 0.8084

Epoch 00002: val_accuracy improved from 0.79441 to 0.80838, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 3/50
63/63 [==============================] - 1s 16ms/step - loss: 0.4037 - accuracy: 0.8160 - val_loss: 0.5026 - val_accuracy: 0.7964

Epoch 00003: val_accuracy did not improve from 0.80838
Epoch 4/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3563 - accuracy: 0.8346 - val_loss: 0.5039 - val_accuracy: 0.7844

Epoch 00004: val_accuracy did not improve from 0.80838
Epoch 5/50
63/63 [==============================] - 1s 16ms/step - loss: 0.3292 - accuracy: 0.8552 - val_loss: 0.5306 - val_accuracy: 0.8104

Epoch 00005: val_accuracy improved from 0.80838 to 0.81038, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\weights.hdf5
Epoch 6/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2989 - accuracy: 0.8607 - val_loss: 0.5253 - val_accuracy: 0.7764

Epoch 00006: val_accuracy did not improve from 0.81038
Epoch 7/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2804 - accuracy: 0.8828 - val_loss: 0.5333 - val_accuracy: 0.7944

Epoch 00007: val_accuracy did not improve from 0.81038
Epoch 8/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2495 - accuracy: 0.8869 - val_loss: 0.5279 - val_accuracy: 0.7864

Epoch 00008: val_accuracy did not improve from 0.81038
Epoch 9/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2454 - accuracy: 0.9077 - val_loss: 0.5420 - val_accuracy: 0.7864

Epoch 00009: val_accuracy did not improve from 0.81038
Epoch 10/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2218 - accuracy: 0.9159 - val_loss: 0.5586 - val_accuracy: 0.7804

Epoch 00010: val_accuracy did not improve from 0.81038
Epoch 11/50
63/63 [==============================] - 1s 15ms/step - loss: 0.2041 - accuracy: 0.9302 - val_loss: 0.5507 - val_accuracy: 0.7764

Epoch 00011: val_accuracy did not improve from 0.81038
Epoch 12/50
63/63 [==============================] - 1s 17ms/step - loss: 0.1777 - accuracy: 0.9380 - val_loss: 0.5723 - val_accuracy: 0.7465

Epoch 00012: val_accuracy did not improve from 0.81038
Epoch 13/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1812 - accuracy: 0.9461 - val_loss: 0.5799 - val_accuracy: 0.7565

Epoch 00013: val_accuracy did not improve from 0.81038
Epoch 14/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1676 - accuracy: 0.9510 - val_loss: 0.5875 - val_accuracy: 0.7465

Epoch 00014: val_accuracy did not improve from 0.81038
Epoch 15/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1485 - accuracy: 0.9644 - val_loss: 0.5936 - val_accuracy: 0.7545

Epoch 00015: val_accuracy did not improve from 0.81038
Epoch 16/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1424 - accuracy: 0.9641 - val_loss: 0.6198 - val_accuracy: 0.7505

Epoch 00016: val_accuracy did not improve from 0.81038
Epoch 17/50
63/63 [==============================] - 1s 16ms/step - loss: 0.1340 - accuracy: 0.9714 - val_loss: 0.6263 - val_accuracy: 0.7565

Epoch 00017: val_accuracy did not improve from 0.81038
Epoch 18/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1283 - accuracy: 0.9718 - val_loss: 0.6423 - val_accuracy: 0.7585

Epoch 00018: val_accuracy did not improve from 0.81038
Epoch 19/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1257 - accuracy: 0.9697 - val_loss: 0.6530 - val_accuracy: 0.7445

Epoch 00019: val_accuracy did not improve from 0.81038
Epoch 20/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1158 - accuracy: 0.9797 - val_loss: 0.6629 - val_accuracy: 0.7505

Epoch 00020: val_accuracy did not improve from 0.81038
Epoch 21/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1111 - accuracy: 0.9808 - val_loss: 0.6612 - val_accuracy: 0.7525

Epoch 00021: val_accuracy did not improve from 0.81038
Epoch 22/50
63/63 [==============================] - 1s 16ms/step - loss: 0.1020 - accuracy: 0.9780 - val_loss: 0.6748 - val_accuracy: 0.7505

Epoch 00022: val_accuracy did not improve from 0.81038
Epoch 23/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0907 - accuracy: 0.9854 - val_loss: 0.6868 - val_accuracy: 0.7525

Epoch 00023: val_accuracy did not improve from 0.81038
Epoch 24/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0907 - accuracy: 0.9899 - val_loss: 0.7120 - val_accuracy: 0.7325

Epoch 00024: val_accuracy did not improve from 0.81038
Epoch 25/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0874 - accuracy: 0.9837 - val_loss: 0.7250 - val_accuracy: 0.7146

Epoch 00025: val_accuracy did not improve from 0.81038
Epoch 26/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0779 - accuracy: 0.9919 - val_loss: 0.7326 - val_accuracy: 0.7465

Epoch 00026: val_accuracy did not improve from 0.81038
Epoch 27/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0692 - accuracy: 0.9925 - val_loss: 0.7357 - val_accuracy: 0.7465

Epoch 00027: val_accuracy did not improve from 0.81038
Epoch 28/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0730 - accuracy: 0.9918 - val_loss: 0.7822 - val_accuracy: 0.7046

Epoch 00028: val_accuracy did not improve from 0.81038
Epoch 29/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0698 - accuracy: 0.9865 - val_loss: 0.7895 - val_accuracy: 0.7305

Epoch 00029: val_accuracy did not improve from 0.81038
Epoch 30/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0638 - accuracy: 0.9921 - val_loss: 0.7858 - val_accuracy: 0.7265

Epoch 00030: val_accuracy did not improve from 0.81038
Epoch 31/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0613 - accuracy: 0.9931 - val_loss: 0.7930 - val_accuracy: 0.7345

Epoch 00031: val_accuracy did not improve from 0.81038
Epoch 32/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0586 - accuracy: 0.9937 - val_loss: 0.8081 - val_accuracy: 0.7485

Epoch 00032: val_accuracy did not improve from 0.81038
Epoch 33/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0554 - accuracy: 0.9921 - val_loss: 0.7990 - val_accuracy: 0.7525

Epoch 00033: val_accuracy did not improve from 0.81038
Epoch 34/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0484 - accuracy: 0.9965 - val_loss: 0.8135 - val_accuracy: 0.7385

Epoch 00034: val_accuracy did not improve from 0.81038
Epoch 35/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0453 - accuracy: 0.9968 - val_loss: 0.8380 - val_accuracy: 0.7405

Epoch 00035: val_accuracy did not improve from 0.81038
Epoch 36/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0495 - accuracy: 0.9945 - val_loss: 0.8192 - val_accuracy: 0.7485

Epoch 00036: val_accuracy did not improve from 0.81038
Epoch 37/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0437 - accuracy: 0.9942 - val_loss: 0.8558 - val_accuracy: 0.7305

Epoch 00037: val_accuracy did not improve from 0.81038
Epoch 38/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0395 - accuracy: 0.9979 - val_loss: 0.8623 - val_accuracy: 0.7186

Epoch 00038: val_accuracy did not improve from 0.81038
Epoch 39/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0393 - accuracy: 0.9975 - val_loss: 0.8667 - val_accuracy: 0.7345

Epoch 00039: val_accuracy did not improve from 0.81038
Epoch 40/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0354 - accuracy: 0.9990 - val_loss: 0.8890 - val_accuracy: 0.7026

Epoch 00040: val_accuracy did not improve from 0.81038
Epoch 41/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0383 - accuracy: 0.9973 - val_loss: 0.8938 - val_accuracy: 0.7246

Epoch 00041: val_accuracy did not improve from 0.81038
Epoch 42/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0356 - accuracy: 0.9992 - val_loss: 0.8997 - val_accuracy: 0.7226

Epoch 00042: val_accuracy did not improve from 0.81038
Epoch 43/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0333 - accuracy: 0.9984 - val_loss: 0.9334 - val_accuracy: 0.7246

Epoch 00043: val_accuracy did not improve from 0.81038
Epoch 44/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0281 - accuracy: 0.9987 - val_loss: 0.9452 - val_accuracy: 0.7046

Epoch 00044: val_accuracy did not improve from 0.81038
Epoch 45/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0330 - accuracy: 0.9990 - val_loss: 0.9412 - val_accuracy: 0.7126

Epoch 00045: val_accuracy did not improve from 0.81038
Epoch 46/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0311 - accuracy: 0.9978 - val_loss: 0.9477 - val_accuracy: 0.7265

Epoch 00046: val_accuracy did not improve from 0.81038
Epoch 47/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0279 - accuracy: 0.9990 - val_loss: 0.9422 - val_accuracy: 0.7246

Epoch 00047: val_accuracy did not improve from 0.81038
Epoch 48/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0253 - accuracy: 0.9996 - val_loss: 0.9854 - val_accuracy: 0.7226

Epoch 00048: val_accuracy did not improve from 0.81038
Epoch 49/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0247 - accuracy: 0.9996 - val_loss: 1.0056 - val_accuracy: 0.7325

Epoch 00049: val_accuracy did not improve from 0.81038
Epoch 50/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0234 - accuracy: 1.0000 - val_loss: 0.9983 - val_accuracy: 0.7126

Epoch 00050: val_accuracy did not improve from 0.81038
KNRM运行时间: 0.92 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\save\KNRM\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\KNRM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-eALvP4MF-1623898674927)(output_72_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-n7qnQz2o-1623898674927)(output_72_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.574884	0.7795	0.490436	0.794411	0.001
2	0.440398	0.7985	0.492095	0.808383	0.001
3	0.396315	0.8200	0.502608	0.796407	0.001
4	0.355102	0.8350	0.503900	0.784431	0.001
5	0.331233	0.8550	0.530610	0.810379	0.001
6	0.308696	0.8630	0.525264	0.776447	0.001
7	0.288832	0.8795	0.533331	0.794411	0.001
8	0.273837	0.8805	0.527850	0.786427	0.001
9	0.246026	0.9020	0.541978	0.786427	0.001
10	0.227715	0.9100	0.558648	0.780439	0.001
11	0.212224	0.9275	0.550749	0.776447	0.001
12	0.196974	0.9305	0.572260	0.746507	0.001
13	0.185532	0.9415	0.579861	0.756487	0.001
14	0.172824	0.9470	0.587466	0.746507	0.001
15	0.161000	0.9560	0.593587	0.754491	0.001
16	0.149717	0.9580	0.619803	0.750499	0.001
17	0.142122	0.9625	0.626306	0.756487	0.001
18	0.134651	0.9645	0.642345	0.758483	0.001
19	0.127208	0.9655	0.653023	0.744511	0.001
20	0.119768	0.9750	0.662865	0.750499	0.001
21	0.113280	0.9770	0.661168	0.752495	0.001
22	0.105297	0.9790	0.674814	0.750499	0.001
23	0.097532	0.9815	0.686752	0.752495	0.001
24	0.093924	0.9875	0.711993	0.732535	0.001
25	0.088030	0.9855	0.725046	0.714571	0.001
26	0.081685	0.9895	0.732556	0.746507	0.001
27	0.077664	0.9895	0.735740	0.746507	0.001
28	0.073690	0.9895	0.782233	0.704591	0.001
29	0.074140	0.9855	0.789516	0.730539	0.001
30	0.068505	0.9890	0.785845	0.726547	0.001
31	0.063642	0.9920	0.792976	0.734531	0.001
32	0.058905	0.9940	0.808054	0.748503	0.001
33	0.057791	0.9930	0.798978	0.752495	0.001
34	0.054806	0.9935	0.813470	0.738523	0.001
35	0.052863	0.9940	0.838032	0.740519	0.001
36	0.052058	0.9940	0.819171	0.748503	0.001
37	0.046994	0.9945	0.855772	0.730539	0.001
38	0.044296	0.9965	0.862320	0.718563	0.001
39	0.042409	0.9960	0.866716	0.734531	0.001
40	0.039556	0.9970	0.889020	0.702595	0.001
41	0.039695	0.9965	0.893792	0.724551	0.001
42	0.038656	0.9980	0.899679	0.722555	0.001
43	0.035125	0.9980	0.933449	0.724551	0.001
44	0.033467	0.9975	0.945224	0.704591	0.001
45	0.034590	0.9980	0.941157	0.712575	0.001
46	0.032396	0.9980	0.947692	0.726547	0.001
47	0.031140	0.9990	0.942183	0.724551	0.001
48	0.027756	0.9995	0.985372	0.722555	0.001
49	0.027366	0.9995	1.005574	0.732535	0.001
50	0.025218	0.9990	0.998327	0.712575	0.001

DUET

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
# 这是啥子
lm_xor = keras.layers.Lambda(_xor_match)([sent1_embed, sent2_embed])
lm_conv = keras.layers.Conv1D(32,30,padding='same',activation='relu')(lm_xor)
lm_conv = keras.layers.Dropout(0.5)(lm_conv)
lm_feat = keras.layers.Reshape((-1,))(lm_conv)
for hidden_size in [32]:
    lm_feat = keras.layers.Dense(hidden_size,activation='relu')(lm_feat)
lm_drop = keras.layers.Dropout(0.5)(lm_feat)
lm_score = keras.layers.Dense(1)(lm_drop)
dm_q_conv = keras.layers.Conv1D(32,3,padding='same',activation='relu')(sent1_embed)
dm_q_conv = keras.layers.Dropout(0.5)(dm_q_conv)
dm_q_mp = keras.layers.MaxPooling1D(pool_size=3)(dm_q_conv)
dm_q_rep = keras.layers.Reshape((-1,))(dm_q_mp)
dm_q_rep = keras.layers.Dense(32)(dm_q_rep)
dm_q_rep = keras.layers.Lambda(lambda x: tf.expand_dims(x, 1))(
    dm_q_rep)
dm_d_conv1 = keras.layers.Conv1D(32,3,padding='same',activation='relu')(sent2_embed)
dm_d_conv1 = keras.layers.Dropout(0.5)(dm_d_conv1)
dm_d_mp = keras.layers.MaxPooling1D(pool_size=3)(dm_d_conv1)
dm_d_conv2 = keras.layers.Conv1D(32, 3,padding='same',activation='relu')(dm_d_mp)
dm_d_conv2 = keras.layers.Dropout(0.5)(dm_d_conv2)
h_dot = keras.layers.Lambda(_hadamard_dot)([dm_q_rep, dm_d_conv2])
dm_feat = keras.layers.Reshape((-1,))(h_dot)
for hidden_size in [64, 32]:
    dm_feat = keras.layers.Dense(hidden_size)(dm_feat)
dm_feat_drop = keras.layers.Dropout(0.5)(dm_feat)
dm_score = keras.layers.Dense(1)(dm_feat_drop)
add = keras.layers.Add()([lm_score, dm_score])
outputs=Dense(1,activation='sigmoid',name="output")(add)
model=Model(inputs=[input1,input2],outputs=outputs,name="DUET")
model.summary()

Model: "DUET"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_4 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
conv1d_11 (Conv1D)              (None, 15, 32)       12320       embedding_4[0][0]                
__________________________________________________________________________________________________
dropout_5 (Dropout)             (None, 15, 32)       0           conv1d_11[0][0]                  
__________________________________________________________________________________________________
conv1d_12 (Conv1D)              (None, 15, 32)       12320       embedding_4[1][0]                
__________________________________________________________________________________________________
max_pooling1d_2 (MaxPooling1D)  (None, 5, 32)        0           dropout_5[0][0]                  
__________________________________________________________________________________________________
dropout_6 (Dropout)             (None, 15, 32)       0           conv1d_12[0][0]                  
__________________________________________________________________________________________________
reshape_1 (Reshape)             (None, 160)          0           max_pooling1d_2[0][0]            
__________________________________________________________________________________________________
max_pooling1d_3 (MaxPooling1D)  (None, 5, 32)        0           dropout_6[0][0]                  
__________________________________________________________________________________________________
dense_25 (Dense)                (None, 32)           5152        reshape_1[0][0]                  
__________________________________________________________________________________________________
conv1d_13 (Conv1D)              (None, 5, 32)        3104        max_pooling1d_3[0][0]            
__________________________________________________________________________________________________
lambda_23 (Lambda)              (None, 15, 15, 128)  0           embedding_4[0][0]                
                                                                 embedding_4[1][0]                
__________________________________________________________________________________________________
lambda_24 (Lambda)              (None, 1, 32)        0           dense_25[0][0]                   
__________________________________________________________________________________________________
dropout_7 (Dropout)             (None, 5, 32)        0           conv1d_13[0][0]                  
__________________________________________________________________________________________________
conv1d_10 (Conv1D)              (None, 15, 15, 32)   122912      lambda_23[0][0]                  
__________________________________________________________________________________________________
lambda_25 (Lambda)              (None, 5, 32)        0           lambda_24[0][0]                  
                                                                 dropout_7[0][0]                  
__________________________________________________________________________________________________
dropout_3 (Dropout)             (None, 15, 15, 32)   0           conv1d_10[0][0]                  
__________________________________________________________________________________________________
reshape_2 (Reshape)             (None, 160)          0           lambda_25[0][0]                  
__________________________________________________________________________________________________
reshape (Reshape)               (None, 7200)         0           dropout_3[0][0]                  
__________________________________________________________________________________________________
dense_26 (Dense)                (None, 64)           10304       reshape_2[0][0]                  
__________________________________________________________________________________________________
dense_23 (Dense)                (None, 32)           230432      reshape[0][0]                    
__________________________________________________________________________________________________
dense_27 (Dense)                (None, 32)           2080        dense_26[0][0]                   
__________________________________________________________________________________________________
dropout_4 (Dropout)             (None, 32)           0           dense_23[0][0]                   
__________________________________________________________________________________________________
dropout_8 (Dropout)             (None, 32)           0           dense_27[0][0]                   
__________________________________________________________________________________________________
dense_24 (Dense)                (None, 1)            33          dropout_4[0][0]                  
__________________________________________________________________________________________________
dense_28 (Dense)                (None, 1)            33          dropout_8[0][0]                  
__________________________________________________________________________________________________
add (Add)                       (None, 1)            0           dense_24[0][0]                   
                                                                 dense_28[0][0]                   
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            2           add[0][0]                        
==================================================================================================
Total params: 677,092
Trainable params: 677,092
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 3s 26ms/step - loss: 0.5843 - accuracy: 0.7687 - val_loss: 0.4710 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\weights.hdf5
Epoch 2/50
63/63 [==============================] - 1s 14ms/step - loss: 0.4494 - accuracy: 0.8254 - val_loss: 0.4809 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 1s 14ms/step - loss: 0.4539 - accuracy: 0.8000 - val_loss: 0.4764 - val_accuracy: 0.8124

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3712 - accuracy: 0.8293 - val_loss: 0.4861 - val_accuracy: 0.8024

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 1s 15ms/step - loss: 0.3177 - accuracy: 0.8645 - val_loss: 0.5183 - val_accuracy: 0.7884

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 1s 14ms/step - loss: 0.2606 - accuracy: 0.8889 - val_loss: 0.5392 - val_accuracy: 0.7705

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 1s 14ms/step - loss: 0.2108 - accuracy: 0.9187 - val_loss: 0.6511 - val_accuracy: 0.7764

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1904 - accuracy: 0.9296 - val_loss: 0.6336 - val_accuracy: 0.7804

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1649 - accuracy: 0.9308 - val_loss: 0.6700 - val_accuracy: 0.7705

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1114 - accuracy: 0.9563 - val_loss: 0.7332 - val_accuracy: 0.7325

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1278 - accuracy: 0.9540 - val_loss: 0.8749 - val_accuracy: 0.7325

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1001 - accuracy: 0.9603 - val_loss: 0.8767 - val_accuracy: 0.7745

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 1s 15ms/step - loss: 0.1035 - accuracy: 0.9632 - val_loss: 0.8983 - val_accuracy: 0.7226

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 1s 14ms/step - loss: 0.1125 - accuracy: 0.9580 - val_loss: 0.9088 - val_accuracy: 0.7265

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0827 - accuracy: 0.9699 - val_loss: 1.0249 - val_accuracy: 0.7625

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0908 - accuracy: 0.9618 - val_loss: 0.9792 - val_accuracy: 0.7525

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0699 - accuracy: 0.9713 - val_loss: 1.0313 - val_accuracy: 0.7325

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0584 - accuracy: 0.9816 - val_loss: 1.1755 - val_accuracy: 0.7485

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0634 - accuracy: 0.9765 - val_loss: 1.2195 - val_accuracy: 0.7725

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0534 - accuracy: 0.9796 - val_loss: 1.2413 - val_accuracy: 0.7465

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0508 - accuracy: 0.9820 - val_loss: 1.4019 - val_accuracy: 0.7545

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0599 - accuracy: 0.9810 - val_loss: 1.2181 - val_accuracy: 0.7545

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0485 - accuracy: 0.9796 - val_loss: 1.3431 - val_accuracy: 0.7824

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0653 - accuracy: 0.9788 - val_loss: 1.3694 - val_accuracy: 0.7705

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0529 - accuracy: 0.9836 - val_loss: 1.4829 - val_accuracy: 0.7924

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0445 - accuracy: 0.9863 - val_loss: 1.4505 - val_accuracy: 0.7545

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0345 - accuracy: 0.9831 - val_loss: 1.6170 - val_accuracy: 0.7665

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0374 - accuracy: 0.9862 - val_loss: 1.4732 - val_accuracy: 0.7365

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0697 - accuracy: 0.9748 - val_loss: 1.3316 - val_accuracy: 0.7705

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0465 - accuracy: 0.9814 - val_loss: 1.4061 - val_accuracy: 0.7745

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0464 - accuracy: 0.9851 - val_loss: 1.4792 - val_accuracy: 0.7465

Epoch 00031: val_accuracy did not improve from 0.81437
Epoch 32/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0287 - accuracy: 0.9921 - val_loss: 1.7669 - val_accuracy: 0.7784

Epoch 00032: val_accuracy did not improve from 0.81437
Epoch 33/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0428 - accuracy: 0.9803 - val_loss: 1.5651 - val_accuracy: 0.7305

Epoch 00033: val_accuracy did not improve from 0.81437
Epoch 34/50
63/63 [==============================] - 1s 17ms/step - loss: 0.0310 - accuracy: 0.9892 - val_loss: 1.9694 - val_accuracy: 0.7665

Epoch 00034: val_accuracy did not improve from 0.81437
Epoch 35/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0405 - accuracy: 0.9876 - val_loss: 1.5619 - val_accuracy: 0.7685

Epoch 00035: val_accuracy did not improve from 0.81437
Epoch 36/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0529 - accuracy: 0.9818 - val_loss: 1.6438 - val_accuracy: 0.7505

Epoch 00036: val_accuracy did not improve from 0.81437
Epoch 37/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0437 - accuracy: 0.9833 - val_loss: 1.5540 - val_accuracy: 0.7385

Epoch 00037: val_accuracy did not improve from 0.81437
Epoch 38/50
63/63 [==============================] - 1s 17ms/step - loss: 0.0212 - accuracy: 0.9926 - val_loss: 1.7939 - val_accuracy: 0.7545

Epoch 00038: val_accuracy did not improve from 0.81437
Epoch 39/50
63/63 [==============================] - 1s 16ms/step - loss: 0.0223 - accuracy: 0.9889 - val_loss: 1.6505 - val_accuracy: 0.7345

Epoch 00039: val_accuracy did not improve from 0.81437
Epoch 40/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0251 - accuracy: 0.9928 - val_loss: 1.8213 - val_accuracy: 0.7285

Epoch 00040: val_accuracy did not improve from 0.81437
Epoch 41/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0232 - accuracy: 0.9898 - val_loss: 1.9106 - val_accuracy: 0.7665

Epoch 00041: val_accuracy did not improve from 0.81437
Epoch 42/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0374 - accuracy: 0.9891 - val_loss: 1.9039 - val_accuracy: 0.7645

Epoch 00042: val_accuracy did not improve from 0.81437
Epoch 43/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0453 - accuracy: 0.9881 - val_loss: 1.7437 - val_accuracy: 0.7625

Epoch 00043: val_accuracy did not improve from 0.81437
Epoch 44/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0278 - accuracy: 0.9929 - val_loss: 1.5498 - val_accuracy: 0.7585

Epoch 00044: val_accuracy did not improve from 0.81437
Epoch 45/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0297 - accuracy: 0.9884 - val_loss: 1.7476 - val_accuracy: 0.7605

Epoch 00045: val_accuracy did not improve from 0.81437
Epoch 46/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0243 - accuracy: 0.9943 - val_loss: 1.4905 - val_accuracy: 0.7525

Epoch 00046: val_accuracy did not improve from 0.81437
Epoch 47/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0212 - accuracy: 0.9938 - val_loss: 1.5048 - val_accuracy: 0.7585

Epoch 00047: val_accuracy did not improve from 0.81437
Epoch 48/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0237 - accuracy: 0.9954 - val_loss: 1.8148 - val_accuracy: 0.7585

Epoch 00048: val_accuracy did not improve from 0.81437
Epoch 49/50
63/63 [==============================] - 1s 14ms/step - loss: 0.0101 - accuracy: 0.9975 - val_loss: 1.9479 - val_accuracy: 0.7725

Epoch 00049: val_accuracy did not improve from 0.81437
Epoch 50/50
63/63 [==============================] - 1s 15ms/step - loss: 0.0287 - accuracy: 0.9905 - val_loss: 1.7721 - val_accuracy: 0.7784

Epoch 00050: val_accuracy did not improve from 0.81437
DUET运行时间: 0.94 mins
INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\save\DUET\assets
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DUET\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-GVTqWmSq-1623898674928)(output_77_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-k1ggkhZC-1623898674928)(output_77_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.546751	0.7935	0.471018	0.814371	0.001
2	0.485875	0.8060	0.480949	0.814371	0.001
3	0.441322	0.8140	0.476435	0.812375	0.001
4	0.384277	0.8240	0.486088	0.802395	0.001
5	0.321170	0.8655	0.518297	0.788423	0.001
6	0.273633	0.8810	0.539247	0.770459	0.001
7	0.215189	0.9125	0.651118	0.776447	0.001
8	0.197219	0.9225	0.633586	0.780439	0.001
9	0.185313	0.9240	0.670023	0.770459	0.001
10	0.136315	0.9405	0.733244	0.732535	0.001
11	0.119038	0.9530	0.874861	0.732535	0.001
12	0.115549	0.9535	0.876669	0.774451	0.001
13	0.105906	0.9575	0.898279	0.722555	0.001
14	0.109583	0.9615	0.908792	0.726547	0.001
15	0.087504	0.9700	1.024870	0.762475	0.001
16	0.084737	0.9655	0.979203	0.752495	0.001
17	0.078465	0.9685	1.031322	0.732535	0.001
18	0.063128	0.9795	1.175452	0.748503	0.001
19	0.069982	0.9735	1.219517	0.772455	0.001
20	0.053760	0.9805	1.241341	0.746507	0.001
21	0.049837	0.9835	1.401922	0.754491	0.001
22	0.063830	0.9800	1.218138	0.754491	0.001
23	0.052452	0.9795	1.343117	0.782435	0.001
24	0.054416	0.9790	1.369412	0.770459	0.001
25	0.049495	0.9820	1.482910	0.792415	0.001
26	0.046076	0.9860	1.450504	0.754491	0.001
27	0.043246	0.9820	1.617050	0.766467	0.001
28	0.047107	0.9830	1.473243	0.736527	0.001
29	0.059975	0.9795	1.331571	0.770459	0.001
30	0.042262	0.9845	1.406110	0.774451	0.001
31	0.038564	0.9855	1.479194	0.746507	0.001
32	0.033506	0.9900	1.766863	0.778443	0.001
33	0.041570	0.9815	1.565071	0.730539	0.001
34	0.033610	0.9885	1.969369	0.766467	0.001
35	0.039869	0.9855	1.561921	0.768463	0.001
36	0.039280	0.9865	1.643804	0.750499	0.001
37	0.046237	0.9825	1.553974	0.738523	0.001
38	0.024144	0.9920	1.793919	0.754491	0.001
39	0.032593	0.9860	1.650468	0.734531	0.001
40	0.027968	0.9910	1.821276	0.728543	0.001
41	0.026392	0.9910	1.910593	0.766467	0.001
42	0.027297	0.9920	1.903870	0.764471	0.001
43	0.040179	0.9870	1.743671	0.762475	0.001
44	0.031724	0.9910	1.549826	0.758483	0.001
45	0.031551	0.9915	1.747630	0.760479	0.001
46	0.032493	0.9920	1.490503	0.752495	0.001
47	0.027788	0.9920	1.504812	0.758483	0.001
48	0.024576	0.9940	1.814816	0.758483	0.001
49	0.016065	0.9950	1.947903	0.772455	0.001
50	0.026085	0.9915	1.772141	0.778443	0.001

DRMMTKS

# def _make_multi_layer_perceptron_layer() -> keras.layers.Layer:
#         # TODO: do not create new layers for a second call
#         if not True:
#             raise AttributeError(
#                 'Parameter `with_multi_layer_perception` not set.')

#         def _wrapper(x):
#             activation = 'relu'
#             for _ in range(3):
#                 x = keras.layers.Dense(128,
#                                        activation=activation)(x)
#             return keras.layers.Dense(64,
#                                       activation=activation)(x)

#         return _wrapper

input1 = Input(name='text_left', shape=(max_len,))
input2 = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size)
sent1_embed=embedding(input1)
sent2_embed=embedding(input2)
atten_mask = tf.not_equal(input1, -1)
atten_mask = tf.cast(atten_mask,K.floatx())
# shape = [B, L, 1]
atten_mask = tf.expand_dims(atten_mask, axis=2)
# shape = [B, L, 1]
attention_probs =attention_layer(sent1_embed, atten_mask)

# Matching histogram of top-k
# shape = [B, L, R]
matching_matrix =Dot(axes=[2, 2], normalize=True)([sent1_embed,sent2_embed])
# shape = [B, L, K]
effective_top_k = min(20,5,300)
matching_topk = Lambda(
    lambda x: tf.nn.top_k(x, k=effective_top_k, sorted=True)[0]
)(matching_matrix)

# Process right input.
# shape = [B, L, 1]
dense_output =_make_multi_layer_perceptron_layer()(matching_topk)

# shape = [B, 1, 1]
dot_score = Dot(axes=[1, 1])(
    [attention_probs, dense_output])

flatten_score = Flatten()(dot_score)
outputs=Dense(1,activation='sigmoid',name="output")(flatten_score)
model=Model(inputs=[input1,input2],outputs=outputs,name="DRMMTKS")
model.summary()

====================
Model: "DRMMTKS"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_5 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
dot_4 (Dot)                     (None, 15, 15)       0           embedding_5[0][0]                
                                                                 embedding_5[1][0]                
__________________________________________________________________________________________________
lambda_26 (Lambda)              (None, 15, 5)        0           dot_4[0][0]                      
__________________________________________________________________________________________________
dense_30 (Dense)                (None, 15, 128)      768         lambda_26[0][0]                  
__________________________________________________________________________________________________
dense_29 (Dense)                (None, 15, 1)        128         embedding_5[0][0]                
__________________________________________________________________________________________________
dense_31 (Dense)                (None, 15, 128)      16512       dense_30[0][0]                   
__________________________________________________________________________________________________
attention_mask (Lambda)         (None, 15, 1)        0           dense_29[0][0]                   
__________________________________________________________________________________________________
dense_32 (Dense)                (None, 15, 128)      16512       dense_31[0][0]                   
__________________________________________________________________________________________________
attention_probs (Lambda)        (None, 15, 1)        0           attention_mask[0][0]             
__________________________________________________________________________________________________
dense_33 (Dense)                (None, 15, 64)       8256        dense_32[0][0]                   
__________________________________________________________________________________________________
dot_5 (Dot)                     (None, 1, 64)        0           attention_probs[0][0]            
                                                                 dense_33[0][0]                   
__________________________________________________________________________________________________
flatten_2 (Flatten)             (None, 64)           0           dot_5[0][0]                      
__________________________________________________________________________________________________
output (Dense)                  (None, 1)            65          flatten_2[0][0]                  
==================================================================================================
Total params: 320,641
Trainable params: 320,641
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
# 这里由于是自己定义模型，所以不能保存格式，一下午全耽误在这里了，妈的
# 保留最好的模型
# mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
#                             monitor='val_accuracy',
#                             verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

# 此模型由于包含自定义的模型，所以暂时不知如何保存
# model.save(os.path.join(path,'save',model.name))
model.save_weights(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

WARNING:tensorflow:Model failed to serialize as JSON. Ignoring... ('Not JSON Serializable:', <KerasTensor: shape=(None, 15, 1) dtype=float32 (created by layer 'tf.expand_dims')>)
Epoch 1/50
63/63 [==============================] - 2s 21ms/step - loss: 5.8493e-04 - accuracy: 0.9998 - val_loss: 4.4134 - val_accuracy: 0.7425
Epoch 2/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0048 - accuracy: 0.9989 - val_loss: 4.4415 - val_accuracy: 0.7066
Epoch 3/50
63/63 [==============================] - 1s 12ms/step - loss: 6.6143e-04 - accuracy: 1.0000 - val_loss: 4.3381 - val_accuracy: 0.7126
Epoch 4/50
63/63 [==============================] - 1s 12ms/step - loss: 7.1693e-04 - accuracy: 0.9996 - val_loss: 4.7279 - val_accuracy: 0.7246
Epoch 5/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0345 - accuracy: 0.9949 - val_loss: 3.7630 - val_accuracy: 0.6667
Epoch 6/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0168 - accuracy: 0.9934 - val_loss: 3.1826 - val_accuracy: 0.6886
Epoch 7/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0016 - accuracy: 0.9999 - val_loss: 3.0072 - val_accuracy: 0.6926
Epoch 8/50
63/63 [==============================] - 1s 12ms/step - loss: 0.0037 - accuracy: 0.9991 - val_loss: 3.1302 - val_accuracy: 0.7166
Epoch 9/50
63/63 [==============================] - 1s 14ms/step - loss: 1.4906e-04 - accuracy: 1.0000 - val_loss: 3.1637 - val_accuracy: 0.7126
Epoch 10/50
63/63 [==============================] - 1s 12ms/step - loss: 1.7446e-04 - accuracy: 1.0000 - val_loss: 3.1810 - val_accuracy: 0.7166
Epoch 11/50
63/63 [==============================] - 1s 12ms/step - loss: 8.6755e-05 - accuracy: 1.0000 - val_loss: 3.2007 - val_accuracy: 0.7146
Epoch 12/50
63/63 [==============================] - 1s 12ms/step - loss: 9.7124e-05 - accuracy: 1.0000 - val_loss: 3.2211 - val_accuracy: 0.7166
Epoch 13/50
63/63 [==============================] - 1s 12ms/step - loss: 6.5392e-05 - accuracy: 1.0000 - val_loss: 3.2383 - val_accuracy: 0.7166
Epoch 14/50
63/63 [==============================] - 1s 12ms/step - loss: 5.3699e-05 - accuracy: 1.0000 - val_loss: 3.2553 - val_accuracy: 0.7206
Epoch 15/50
63/63 [==============================] - 1s 12ms/step - loss: 5.7876e-05 - accuracy: 1.0000 - val_loss: 3.2714 - val_accuracy: 0.7206
Epoch 16/50
63/63 [==============================] - 1s 12ms/step - loss: 4.5222e-05 - accuracy: 1.0000 - val_loss: 3.2849 - val_accuracy: 0.7206
Epoch 17/50
63/63 [==============================] - 1s 12ms/step - loss: 4.4179e-05 - accuracy: 1.0000 - val_loss: 3.3002 - val_accuracy: 0.7206
Epoch 18/50
63/63 [==============================] - 1s 12ms/step - loss: 4.1445e-05 - accuracy: 1.0000 - val_loss: 3.3130 - val_accuracy: 0.7206
Epoch 19/50
63/63 [==============================] - 1s 12ms/step - loss: 4.2989e-05 - accuracy: 1.0000 - val_loss: 3.3269 - val_accuracy: 0.7226
Epoch 20/50
63/63 [==============================] - 1s 12ms/step - loss: 3.6369e-05 - accuracy: 1.0000 - val_loss: 3.3399 - val_accuracy: 0.7226
Epoch 21/50
63/63 [==============================] - 1s 12ms/step - loss: 3.4277e-05 - accuracy: 1.0000 - val_loss: 3.3522 - val_accuracy: 0.7226
Epoch 22/50
63/63 [==============================] - 1s 12ms/step - loss: 2.9918e-05 - accuracy: 1.0000 - val_loss: 3.3644 - val_accuracy: 0.7226
Epoch 23/50
63/63 [==============================] - 1s 12ms/step - loss: 2.2931e-05 - accuracy: 1.0000 - val_loss: 3.3751 - val_accuracy: 0.7226
Epoch 24/50
63/63 [==============================] - 1s 12ms/step - loss: 2.2884e-05 - accuracy: 1.0000 - val_loss: 3.3872 - val_accuracy: 0.7226
Epoch 25/50
63/63 [==============================] - 1s 12ms/step - loss: 3.1830e-05 - accuracy: 1.0000 - val_loss: 3.3990 - val_accuracy: 0.7226
Epoch 26/50
63/63 [==============================] - 1s 12ms/step - loss: 2.1965e-05 - accuracy: 1.0000 - val_loss: 3.4101 - val_accuracy: 0.7226
Epoch 27/50
63/63 [==============================] - 1s 12ms/step - loss: 2.0477e-05 - accuracy: 1.0000 - val_loss: 3.4210 - val_accuracy: 0.7226
Epoch 28/50
63/63 [==============================] - 1s 12ms/step - loss: 1.4543e-05 - accuracy: 1.0000 - val_loss: 3.4321 - val_accuracy: 0.7226
Epoch 29/50
63/63 [==============================] - 1s 12ms/step - loss: 1.8529e-05 - accuracy: 1.0000 - val_loss: 3.4380 - val_accuracy: 0.7246
Epoch 30/50
63/63 [==============================] - 1s 12ms/step - loss: 1.8538e-05 - accuracy: 1.0000 - val_loss: 3.4432 - val_accuracy: 0.7246
Epoch 31/50
63/63 [==============================] - 1s 12ms/step - loss: 1.9024e-05 - accuracy: 1.0000 - val_loss: 3.4492 - val_accuracy: 0.7246
Epoch 32/50
63/63 [==============================] - 1s 12ms/step - loss: 1.5145e-05 - accuracy: 1.0000 - val_loss: 3.4547 - val_accuracy: 0.7246
Epoch 33/50
63/63 [==============================] - 1s 12ms/step - loss: 1.6403e-05 - accuracy: 1.0000 - val_loss: 3.4603 - val_accuracy: 0.7246
DRMMTKS运行时间: 0.5 mins
保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\DRMMTKS\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-el6np97w-1623898674928)(output_83_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-jtD73anI-1623898674929)(output_83_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	0.001804	0.9995	4.413449	0.742515	0.0010
2	0.009143	0.9975	4.441485	0.706587	0.0010
3	0.000664	1.0000	4.338068	0.712575	0.0010
4	0.001877	0.9990	4.727859	0.724551	0.0010
5	0.040458	0.9935	3.763010	0.666667	0.0010
6	0.021687	0.9920	3.182616	0.688623	0.0010
7	0.002447	0.9995	3.007156	0.692615	0.0010
8	0.002465	0.9995	3.130153	0.716567	0.0010
9	0.000166	1.0000	3.163674	0.712575	0.0010
10	0.000125	1.0000	3.180999	0.716567	0.0010
11	0.000101	1.0000	3.200660	0.714571	0.0010
12	0.000086	1.0000	3.221150	0.716567	0.0010
13	0.000075	1.0000	3.238322	0.716567	0.0010
14	0.000067	1.0000	3.255303	0.720559	0.0010
15	0.000060	1.0000	3.271362	0.720559	0.0010
16	0.000053	1.0000	3.284854	0.720559	0.0010
17	0.000048	1.0000	3.300204	0.720559	0.0010
18	0.000043	1.0000	3.313007	0.720559	0.0010
19	0.000039	1.0000	3.326913	0.722555	0.0010
20	0.000036	1.0000	3.339939	0.722555	0.0010
21	0.000033	1.0000	3.352233	0.722555	0.0010
22	0.000031	1.0000	3.364448	0.722555	0.0010
23	0.000028	1.0000	3.375112	0.722555	0.0010
24	0.000026	1.0000	3.387231	0.722555	0.0010
25	0.000024	1.0000	3.399038	0.722555	0.0010
26	0.000023	1.0000	3.410101	0.722555	0.0010
27	0.000021	1.0000	3.420998	0.722555	0.0010
28	0.000020	1.0000	3.432114	0.722555	0.0010
29	0.000019	1.0000	3.438015	0.724551	0.0005
30	0.000018	1.0000	3.443197	0.724551	0.0005
31	0.000017	1.0000	3.449170	0.724551	0.0005
32	0.000017	1.0000	3.454719	0.724551	0.0005
33	0.000016	1.0000	3.460270	0.724551	0.0005

MVLSTM

query= Input(name='text_left', shape=(max_len,))
doc  = Input(name='text_right', shape=(max_len,))
embedding = Embedding(vocab_size, embedding_size,mask_zero=True)
embed_query=embedding(query)
embed_doc=embedding(doc)
# Bi-directional LSTM layer
rep_query = keras.layers.Bidirectional(keras.layers.LSTM(
32,return_sequences=True,dropout=0.2
))(embed_query)
rep_doc = keras.layers.Bidirectional(keras.layers.LSTM(
32,return_sequences=True,dropout=0.2
))(embed_doc)
# Top-k matching layer
matching_matrix = keras.layers.Dot(
    axes=[2, 2], normalize=False)([rep_query, rep_doc])
matching_signals = keras.layers.Reshape((-1,))(matching_matrix)
matching_topk = keras.layers.Lambda(
lambda x: tf.nn.top_k(x, k=10, sorted=True)[0]
)(matching_signals)
# mlp = _make_multi_layer_perceptron_layer()(matching_topk)
mlp =_make_multi_layer_perceptron_layer()(matching_topk)
mlp = keras.layers.Dropout(rate=0.2)(mlp)

x_out = Dense(1, activation='linear')(mlp)
# self._backend = keras.Model(inputs=[query, doc], outputs=x_out)
model=Model(inputs=[query, doc],outputs=x_out,name="MVLSTM")
model.summary()

Model: "MVLSTM"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
text_left (InputLayer)          [(None, 15)]         0                                            
__________________________________________________________________________________________________
text_right (InputLayer)         [(None, 15)]         0                                            
__________________________________________________________________________________________________
embedding_6 (Embedding)         (None, 15, 128)      278400      text_left[0][0]                  
                                                                 text_right[0][0]                 
__________________________________________________________________________________________________
bidirectional (Bidirectional)   (None, 15, 64)       41216       embedding_6[0][0]                
__________________________________________________________________________________________________
bidirectional_1 (Bidirectional) (None, 15, 64)       41216       embedding_6[1][0]                
__________________________________________________________________________________________________
dot_6 (Dot)                     (None, 15, 15)       0           bidirectional[0][0]              
                                                                 bidirectional_1[0][0]            
__________________________________________________________________________________________________
reshape_3 (Reshape)             (None, 225)          0           dot_6[0][0]                      
__________________________________________________________________________________________________
lambda_27 (Lambda)              (None, 10)           0           reshape_3[0][0]                  
__________________________________________________________________________________________________
dense_34 (Dense)                (None, 128)          1408        lambda_27[0][0]                  
__________________________________________________________________________________________________
dense_35 (Dense)                (None, 128)          16512       dense_34[0][0]                   
__________________________________________________________________________________________________
dense_36 (Dense)                (None, 128)          16512       dense_35[0][0]                   
__________________________________________________________________________________________________
dense_37 (Dense)                (None, 64)           8256        dense_36[0][0]                   
__________________________________________________________________________________________________
dropout_9 (Dropout)             (None, 64)           0           dense_37[0][0]                   
__________________________________________________________________________________________________
dense_38 (Dense)                (None, 1)            65          dropout_9[0][0]                  
==================================================================================================
Total params: 403,585
Trainable params: 403,585
Non-trainable params: 0
__________________________________________________________________________________________________

model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

# 各种callback函数
tb_callback = tf.keras.callbacks.TensorBoard(logdir, histogram_freq=1)
#如果accuracy在25个epoch后没有提升，学习率减半。
lr_callback = tf.keras.callbacks.ReduceLROnPlateau(monitor="accuracy",factor = 0.5, patience = 25)
#当accuracy在30个epoch后没有提升，则提前终止训练。
stop_callback = tf.keras.callbacks.EarlyStopping(monitor = "accuracy", patience= 30)

#获取当前的地址
path=os.path.join(os.getcwd(),'outputs\model',model.name)

import os
dirs = path

if not os.path.exists(dirs):
    os.makedirs(dirs)

    
    
# 保留最好的模型
mc_callback=tf.keras.callbacks.ModelCheckpoint(filepath=os.path.join(path,'weights.hdf5'),
                            monitor='val_accuracy',
                            verbose=1, save_best_only=True)


# 把训练轮结果数据流到 csv 文件的回调函数。
# url=os.path.join(os.getcwd(),'outputs')


csv_callback=tf.keras.callbacks.CSVLogger(os.path.join(path,'training.log'))
 
callbacks_list = [tb_callback,lr_callback,stop_callback,mc_callback,csv_callback]

t = time()
history=model.fit(x=[sent1_train,sent2_train],y=train_labels,validation_data=([sent1_val,sent2_val],val_labels),epochs=50,\
          callbacks = callbacks_list)
print(model.name+'运行时间: {} mins'.format(round((time() - t) / 60, 2)))

model.save(os.path.join(path,'save',model.name))
print('保存成功','保存地址在',os.path.join(path,'save'))

Epoch 1/50
63/63 [==============================] - 36s 433ms/step - loss: 3.1179 - accuracy: 0.7979 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00001: val_accuracy improved from -inf to 0.81437, saving model to H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\weights.hdf5
Epoch 2/50
63/63 [==============================] - 23s 364ms/step - loss: 2.9303 - accuracy: 0.8100 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00002: val_accuracy did not improve from 0.81437
Epoch 3/50
63/63 [==============================] - 23s 371ms/step - loss: 2.9783 - accuracy: 0.8069 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00003: val_accuracy did not improve from 0.81437
Epoch 4/50
63/63 [==============================] - 23s 371ms/step - loss: 3.0006 - accuracy: 0.8055 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00004: val_accuracy did not improve from 0.81437
Epoch 5/50
63/63 [==============================] - 23s 373ms/step - loss: 2.8179 - accuracy: 0.8173 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00005: val_accuracy did not improve from 0.81437
Epoch 6/50
63/63 [==============================] - 23s 372ms/step - loss: 3.0154 - accuracy: 0.8045 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00006: val_accuracy did not improve from 0.81437
Epoch 7/50
63/63 [==============================] - 23s 372ms/step - loss: 3.1732 - accuracy: 0.7943 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00007: val_accuracy did not improve from 0.81437
Epoch 8/50
63/63 [==============================] - 24s 374ms/step - loss: 3.0198 - accuracy: 0.8042 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00008: val_accuracy did not improve from 0.81437
Epoch 9/50
63/63 [==============================] - 23s 369ms/step - loss: 3.0138 - accuracy: 0.8046 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00009: val_accuracy did not improve from 0.81437
Epoch 10/50
63/63 [==============================] - 24s 376ms/step - loss: 2.9997 - accuracy: 0.8055 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00010: val_accuracy did not improve from 0.81437
Epoch 11/50
63/63 [==============================] - 23s 373ms/step - loss: 2.9870 - accuracy: 0.8064 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00011: val_accuracy did not improve from 0.81437
Epoch 12/50
63/63 [==============================] - 23s 368ms/step - loss: 2.9587 - accuracy: 0.8082 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00012: val_accuracy did not improve from 0.81437
Epoch 13/50
63/63 [==============================] - 23s 366ms/step - loss: 2.9547 - accuracy: 0.8084 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00013: val_accuracy did not improve from 0.81437
Epoch 14/50
63/63 [==============================] - 23s 368ms/step - loss: 2.7369 - accuracy: 0.8226 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00014: val_accuracy did not improve from 0.81437
Epoch 15/50
63/63 [==============================] - 23s 363ms/step - loss: 3.0141 - accuracy: 0.8046 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00015: val_accuracy did not improve from 0.81437
Epoch 16/50
63/63 [==============================] - 23s 362ms/step - loss: 3.0549 - accuracy: 0.8019 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00016: val_accuracy did not improve from 0.81437
Epoch 17/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9716 - accuracy: 0.8073 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00017: val_accuracy did not improve from 0.81437
Epoch 18/50
63/63 [==============================] - 23s 363ms/step - loss: 2.9648 - accuracy: 0.8078 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00018: val_accuracy did not improve from 0.81437
Epoch 19/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9977 - accuracy: 0.8057 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00019: val_accuracy did not improve from 0.81437
Epoch 20/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9069 - accuracy: 0.8115 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00020: val_accuracy did not improve from 0.81437
Epoch 21/50
63/63 [==============================] - 23s 363ms/step - loss: 2.7864 - accuracy: 0.8194 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00021: val_accuracy did not improve from 0.81437
Epoch 22/50
63/63 [==============================] - 23s 363ms/step - loss: 3.0828 - accuracy: 0.8001 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00022: val_accuracy did not improve from 0.81437
Epoch 23/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9424 - accuracy: 0.8092 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00023: val_accuracy did not improve from 0.81437
Epoch 24/50
63/63 [==============================] - 23s 364ms/step - loss: 3.1591 - accuracy: 0.7952 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00024: val_accuracy did not improve from 0.81437
Epoch 25/50
63/63 [==============================] - 23s 364ms/step - loss: 3.0106 - accuracy: 0.8048 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00025: val_accuracy did not improve from 0.81437
Epoch 26/50
63/63 [==============================] - 23s 363ms/step - loss: 2.9116 - accuracy: 0.8112 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00026: val_accuracy did not improve from 0.81437
Epoch 27/50
63/63 [==============================] - 23s 362ms/step - loss: 2.9635 - accuracy: 0.8079 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00027: val_accuracy did not improve from 0.81437
Epoch 28/50
63/63 [==============================] - 23s 362ms/step - loss: 3.1087 - accuracy: 0.7985 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00028: val_accuracy did not improve from 0.81437
Epoch 29/50
63/63 [==============================] - 23s 361ms/step - loss: 3.0865 - accuracy: 0.7999 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00029: val_accuracy did not improve from 0.81437
Epoch 30/50
63/63 [==============================] - 23s 362ms/step - loss: 3.0061 - accuracy: 0.8051 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00030: val_accuracy did not improve from 0.81437
Epoch 31/50
63/63 [==============================] - 23s 362ms/step - loss: 2.8798 - accuracy: 0.8133 - val_loss: 2.8633 - val_accuracy: 0.8144

Epoch 00031: val_accuracy did not improve from 0.81437
MVLSTM运行时间: 12.2 mins


WARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_4_layer_call_fn while saving (showing 5 of 20). These functions will not be directly callable after loading.
WARNING:absl:Found untraced functions such as lstm_cell_1_layer_call_fn, lstm_cell_1_layer_call_and_return_conditional_losses, lstm_cell_2_layer_call_fn, lstm_cell_2_layer_call_and_return_conditional_losses, lstm_cell_4_layer_call_fn while saving (showing 5 of 20). These functions will not be directly callable after loading.


INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save\MVLSTM\assets


INFO:tensorflow:Assets written to: H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save\MVLSTM\assets


保存成功 保存地址在 H:\code\BNLP学习\NLP项目教程\匹配\文本匹配\花呗问答匹配\outputs\model\MVLSTM\save

plot_metric(history,"loss")
plot_metric(history,"accuracy")

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-AX7pRTRt-1623898674929)(output_88_0.svg)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-XjHSJbUO-1623898674930)(output_88_1.svg)]

# 以表格形式展示
dfhistory = pd.DataFrame(history.history)
dfhistory.index = range(1,len(dfhistory) + 1)
dfhistory.index.name = 'epoch'
dfhistory.to_csv(os.path.join(path,'save',model.name,model.name+'.csv'))
 
dfhistory

	loss	accuracy	val_loss	val_accuracy	lr
epoch
1	2.992440	0.806	2.863314	0.814371	0.0010
2	2.992440	0.806	2.863314	0.814371	0.0010
3	2.992439	0.806	2.863314	0.814371	0.0010
4	2.992440	0.806	2.863314	0.814371	0.0010
5	2.992440	0.806	2.863314	0.814371	0.0010
6	2.992440	0.806	2.863314	0.814371	0.0010
7	2.992440	0.806