一:Self Consistency
题目: SELF-CONSISTENCY IMPROVES CHAIN OF THOUGHT REASONING IN LANGUAGE MODELS
机构:Google Brain, ICLR 2023
论文: https://arxiv.org/pdf/2203.11171.pdf
任务: 对于复杂问题而言,往往可以从多条推理路径得到最终的答案,因此将原来的CoT贪心解码进行优化,提出一种Self Consistency的解码算法
特点: sample-and-marginalize,投票,能够避免CoT的解码的局部最优以及输出重复,可以视作一种“self-ensemble”,无需训练/标注/微调,很容易与现存的采样算法,比如 temperature sampling, top-k sampling,nucleus sampling即插即用。
前置相关工作:CoT