【指令微调数据集生成】Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation

Learning to Generate Instruction Tuning Datasets for Zero-Shot Task Adaptation 学习生成零镜头任务适应的指令调整数据集

论文地址
代码地址
数据集地址

Abstract

We introduce Bonito, an open-source model for conditional task generation: the task of converting unannotated text into task-specific training datasets for instruction tuning. Our goal is to enable zero-shot task adaptation of large language models on users’ specialized, private data. We train Bonito on a new large-scale dataset with 1.65M examples created by remixing existing instruction tuning datasets into meta-templates. The meta-templates for a dataset produce training examples where the input is the unannotated text and the task attribute and the output consists of the instruction and the response. We use Bonito to generate synthetic tasks for seven datasets from specialized domains across three task types – yes-no question answering, extractive question answering, and natural language inference – and adapt language models. We show that Bonito significantly improves the average performance of pretrained and instruction tuned models over the de facto self supervised baseline. For example, adapting Mistral-Instruct-v2 and instruction tuned variants of Mistral and Llama2 with Bonito improves the strong zero-shot performance by 22.1 F1 points whereas the next word prediction objective undoes some of the benefits of instruction tuning and reduces the average performance by 0.8 F1 points. We conduct additional experiments with Bonito to understand the effects of the domain, the size of the training set, and the choice of alternative synthetic task generators. Overall, we show that learning with synthetic instruction tuning datasets is an effective way to adapt language models to new domains. The model, dataset, and code are available at https://github.com/BatsResearch/bonito.

Bonito，是一个用于条件任务生成的开源模型：将未注明的文本转换为特定任务的训练数据集，以便进行指令调整。

我们的目标是在用户的专用私人数据上实现大型语言模型的零样本任务调整。

我们在一个新的大规模数据集上对 Bonito 进行了训练，该数据集包含 165 万个示例，是通过将现有的指令调整数据集重新混合成元模板而创建的。

数据集的元模板可生成训练示例：

• 输入：未注明的文本和任务属性；
• 输出：包括指令和响应。

我们使用 Bonito 为七个数据集生成合成任务（这些数据集来自三个任务类型：是非题QA、提取题QA和自然语言推理）并调整语言模型。

我们的研究表明，与事实上的自我监督基线相比，Bonito 显著提高了预训练和指令调整模型的平均性能。

• 使用 Bonito 对 Mistral-Instruct-v2 以及 Mistral 和 Llama2 的指令调整变体进行调整，可将零样本性能 F1 分数提高 22.1；
• 下一个单词预测目标则抵消了指令调整的部分优势，并将平均性能F1分数降低了 0.8 。

我们还利用 Bonito 进行了其他实验，以了解领域、训练集大小以及选择其他合成任务生成器的影响。

总之，我们证明了使用合成指令调整数据集进行学习是使语言模型适应新领域的有效方法。

Results