CFCRS实验

1.Requirments

python=3.8
pytorch=1.8.1
cudatoolkit=11.1.1
pyg=2.0.1
transformers=4.21.3
accelerate=0.12
nltk=3.6

只列举一些关键的包。

硬件： 24G gpu

数据集： redial

2.Training Recommendation Dialogue Simulator

这里需要修改train_FLM.sh,将sample_path.py生成的文件复制到相应flow文件夹下:

python sample_path.py --n_walks 10000
cd ../../
# redial,inspired
cp data/redial-flow/path_num-10000.jsonl model/simulator/data/${dataset}_flow

网不好的记得在训练之前先提前下载好facebook/bart-base.

接下来运行train_schema.sh.

2.1 FLM实验结果：

Summary:

bleu-1	bleu-2	bleu-3	bleu-4	dist-1	dist-2	dist-3	dist-4	loss
0.88	0.21	0.06	0.22	2.37	8.10	8.94	8.07	3.19

2.2 Schema实验结果：

Summary:

recall@1	recall@5	recall@10	loss
0.49	0.99	0.84	3.31

3.Training CRS models

这里只使用UniCRS作为base model.

bash script/UniCRS/redial/train_pre.sh 0 
bash script/UniCRS/redial/train_rec.sh 0 
bash script/UniCRS/redial/train_cf.sh 0 
bash script/UniCRS/redial/train_conv.sh 0 

3.1 train_rec结果：

3.2 train_cf结果：

Summary:

model	recall@1	recall@10	recall@50	mrr@1	mrr@10	mrr@50	ndcg@1	ndcg@10	ndcg@50
UniCRS	0.047	0.221	0.430	0.047	0.092	0.101	0.047	0.122	0.167
UniCRS-CFCRS	0.048	0.232	0.447	0.048	0.093	0.103	0.048	0.126	0.172

论文结果：

3.3 train_conv结果：

bleu-1	bleu-2	bleu-3	bleu-4	dist-1	dist-2	dist-3	dist-4
0.186	0.039	0.016	0.009	0.0651	2.453	4.270	5.224