相似度论文再回顾

Towards a Unified Multi-Dimensional Evaluator for Text Generation

多个维度出发评价生成文本的质量，如一致性、流畅度等等。

每个维度的伪标注样本数量为30K，作者构建的数据集：

we first design specific rules for several commonly evaluated dimensions to construct pseudo data, and then combine them to train the evaluator.

任务形式：summary和dialogue。

实验验证：对比model有BLEU、METHOR、ROUGE、Bertscore....

人工标注的数据：TO verfify the proposed evaluator is qualifited, we need to calculated correlations with human scores in each benchamark.

Train the evaluator for 1-3 epochs. _Supervised method.

BARTSCORE: Evaluating Generated Text as Text Generation

Conditional text generation: for example,machine translation, so the goal is to generate a hypothesis (h = h1, · · · , hm) based on a given source text (s = s1, · · · , sn)

require human judgments to train (i.e., supervised metrics): COMET [57], BLEURT [63], or are human judgment-free (i.e., unsupervised): BLEU [51] ROUGE-1 and ROUGE-2, ROUGE-L, CHRF [53], PRISM [66], MoverScore [77], BERTScore [76].

Datasets (such are generate for specific areas from 2015):

TASK	Datasets	Descrip
SUM	NER 18	60个articles
MT	WMT 19
Factuality	Rank19	373 triples of a source sentence with two summary sentences, one correct and one incorrect.
Factuality	QAGS20	235 test outputs on CNNDM dataset from [16] and 239 test outputs on XSUM dataset [48] from BART fine-tuned on XSUM
Data to Text	BAGEL	202 samples , each sample consists of one meaning representation, multiple references, and utterances generated by different systems

Train the evaluator .

BERTSCORE: EVALUATING TEXT GENERATION WITH BERT

dataset: wnt 18和wnt 19

task format :machine translation and image capition

No training.