浦语书生大模型·llamaindex+Internlm2 RAG实践

问题：xtuner是什么？

1.创建新的开发机及环境配置

1.配置环境

conda create -n llamaindex python=3.10

2.激活环境

conda activate llamaindex

3.安装相关基础依赖 python 虚拟环境

pip install llama-index==0.10.38 llama-index-llms-huggingface==0.2.0 "transformers[torch]==4.41.1" "huggingface_hub[inference]==0.23.1" huggingface_hub==0.23.1 sentence-transformers==2.7.0 sentencepiece==0.2.0

4.安装 Llamaindex和相关的包

pip install llama-index==0.10.38 llama-index-llms-huggingface==0.2.0 "transformers[torch]==4.41.1" "huggingface_hub[inference]==0.23.1" huggingface_hub==0.23.1 sentence-transformers==2.7.0 sentencepiece==0.2.0

5.下载 Sentence Transformer 模型

cd ~
mkdir llamaindex_demo
mkdir model
cd ~/llamaindex_demo
touch download_hf.py

打开download_hf.py 贴入以下代码

import os
# 设置环境变量
os.environ['HF_ENDPOINT'] = 'https://hf-mirror.com'
# 下载模型
os.system('huggingface-cli download --resume-download sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2 --local-dir /root/model/sentence-transformer')

6.下载 NLTK 相关资源 

cd /root
git clone https://gitee.com/yzy0612/nltk_data.git  --branch gh-pages
cd nltk_data
mv packages/*  ./
cd tokenizers
unzip punkt.zip
cd ../taggers
unzip averaged_perceptron_tagger.zip

2.LlamaIndex HuggingFaceLLM

链接模型

cd ~/model
ln -s /root/share/new_models/Shanghai_AI_Laboratory/internlm2-chat-1_8b/ ./
新建一个python文件，并放入下面Python代码
cd ~/llamaindex_demo
touch llamaindex_internlm.py

from llama_index.llms.huggingface import HuggingFaceLLM
from llama_index.core.llms import ChatMessage
llm = HuggingFaceLLM(
    model_name="/root/model/internlm2-chat-1_8b",
    tokenizer_name="/root/model/internlm2-chat-1_8b",
    model_kwargs={"trust_remote_code":True},
    tokenizer_kwargs={"trust_remote_code":True}
)
 
rsp = llm.chat(messages=[ChatMessage(content="xtuner是什么？")])
print(rsp)

运行

conda activate llamaindex
cd ~/llamaindex_demo/
python llamaindex_internlm.py

不使用RAG的回复如下：结果表明它并不能回答

3.LlamaIndex RAG

from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
 
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.huggingface import HuggingFaceLLM
 
embed_model = HuggingFaceEmbedding(
    model_name="/root/model/sentence-transformer"
)
 
Settings.embed_model = embed_model
 
llm = HuggingFaceLLM(
    model_name="/root/model/internlm2-chat-1_8b",
    tokenizer_name="/root/model/internlm2-chat-1_8b",
    model_kwargs={"trust_remote_code":True},
    tokenizer_kwargs={"trust_remote_code":True}
)
Settings.llm = llm
 
documents = SimpleDirectoryReader("/root/llamaindex_demo/data").load_data()
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
response = query_engine.query("xtuner是什么?")
 
print(response)

使用RAG后的回复如下所示：