【LLMs组合合成 进化算法】Evolutionary Optimization of Model Merging Recipes

Evolutionary Optimization of Model Merging Recipes 模型合并配方的进化优化

2024.3.19

论文地址
代码地址

Abstract

We present a novel application of evolutionary algorithms to automate the creation of powerful foundation models. While model merging has emerged as a promising approach for LLM development due to its cost-effectiveness, it currently relies on human intuition and domain knowledge, limiting its potential. Here, we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing for optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models like a Japanese LLM with Math reasoning capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese VLMs. This work not only contributes new state-of-the-art models back to the open-source community, but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.

我们介绍了进化算法在自动创建强大基础模型方面的新应用。

虽然模型合并因其成本效益而成为一种有前途的 LLM 开发方法，但它目前依赖于人类的直觉和领域知识，限制了其潜力。

在这里，我们提出了一种进化方法，通过自动发现不同开源模型的有效组合来克服这一局限，利用它们的集体智慧，而无需大量额外的训练数据或计算。我们的方法可在参数空间和数据流空间中运行，从而实现超越单个模型权重的优化。

这种方法甚至还能促进跨领域合并，生成类似具有数学推理能力的日语 LLM 这样的模型。

• 令人惊讶的是，我们的日语数学 LLM 在各种既定的日语 LLM 基准测试中取得了一流的性能，甚至超过了参数明显更多的模型，尽管这些模型并没有针对此类任务进行明确的训练。

• 此外，通过我们的方法生成的具有文化感知能力的日语 VLM 在描述日本文化特定内容方面也证明了其有效性，表现优于之前的日语 VLM。

这项工作不仅为开源社区贡献了新的先进模型，还为自动模型组成引入了一种新的范式，为探索其他高效的方法以实现更高的效率铺平了道路。

Models

注：详见项目地址

Results