progressive-generation-master代码记录【下载处理数据】（添加缺失字符）

def _add_missing_period(line):
    END_TOKENS = [".", "!", "?", "...", "'", "`", '"', "\u2019", "\u2019", ")"]
    if line.startswith("@highlight"):
        return line
    if line[-1] in END_TOKENS:
        return line
    return line + "."

END_TOKENS = [".", "!", "?", "...", "'", "`", '"', "\u2019", "\u2019", ")"]

检测每个句子是否缺失符号，如果是上述的字符，将后面补充上.符号