概念解析 | LoRA:低秩矩阵分解在神经网络微调中的作用

注1:本文系“概念解析”系列之一,致力于简洁清晰地解释、辨析复杂而专业的概念。本次辨析的概念是:基于低秩矩阵分解的神经网络微调方法LoRA

LoRA: Low-Rank Adaptation of Large Language Models

LoRA由如下论文提出，详细信息请参见论文原文
https://arxiv.org/abs/2106.09685

@inproceedings{
hu2022lora,
title={Lo{RA}: Low-Rank Adaptation of Large Language Models},
author={Edward J Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen},
booktitle={International Conference on Learning Representations},
year={2022},
url={https://openreview.net/for