智能驾道：RAG助我成就车界传奇

项目名称：基于RAG的对于特定汽车的助手

报告日期：2024年8月18日

项目负责人：黄浩宇

### 项目概述

本项目的目标是开发一个基于RAG技术的文字智能对话机器人，旨在帮助用户解决使用车辆时遇到的问题，尤其是涉及导航系统的使用、功能设置等方面。通过结合RAG模型的强大信息检索能力和交互功能，用户能够更加便捷地获取所需信息，提高行车安全性和便利性。

项目的应用场景包括：

1. **导航与路线规划**：帮助用户通过语音或文本输入快速获取驾驶路线。

2. **车载系统功能查询**：用户可以通过对话机器人了解车辆功能的详细信息和操作指南。

3. **实时帮助与故障排查**：在行车过程中，用户可以实时获取解决车辆故障或使用问题的指导。

### 技术方案与实施步骤

#### 模型选择：本项目采用了Meta公司的`meta/llama-3.1-405b-instruct`模型，结合RAG（Retrieval-Augmented Generation）技术。选择该模型的理由包括：

- **高效的信息检索能力**：RAG模型能够通过检索外部文档或数据库，生成更加精确和上下文相关的回答。

- **强大的生成能力**：`meta/llama-3.1-405b-instruct`模型具备强大的自然语言生成能力，能够理解复杂的查询并提供准确的回答。

- **灵活的集成**：RAG模型能够与不同的数据源灵活集成，如车辆用户手册和实时导航数据，为用户提供全面的支持。

#### 数据的构建

数据构建过程中，我们主要使用了两份关于Honda HR-V车辆的用户手册，对这些文档进行了向量化处理。数据向量化的方法主要包括：

- **使用词向量模型**：对手册中的文本进行词向量化，以便RAG模型能够高效地检索相关信息。

- **构建知识库**：将向量化后的数据存储在一个知识库中，供RAG模型进行检索。通过这种方式，模型可以实时提供与用户查询相关的精确信息。

#### 实施步骤：

1. **环境搭建**： 

·  安装必要的软件，包括Python、PyTorch等机器学习库。

·  配置开发环境，确保模型训练和推理能够顺利进行。

·  部署语音识别和自然语言处理工具，如Whisper模型，用于语音输入的处理。

2. **代码实现**： 

Step 1 - 使用NVIDIA_API_KEY：

import getpass

import os

if os.environ.get("NVIDIA_API_KEY", "").startswith("nvapi-"):

    print("Valid NVIDIA_API_KEY already in environment. Delete to reset")

else:

    nvapi_key = getpass.getpass("NVAPI Key (starts with nvapi-): ")

    assert nvapi_key.startswith("nvapi-"), f"{nvapi_key[:5]}... is not a valid key"

os.environ["NVIDIA_API_KEY"] = nvapi_key

from langchain_nvidia_ai_endpoints import ChatNVIDIA

ChatNVIDIA.get_available_models()

Step 2 - 初始化SLM

这里我们使用 meta/llama-3.1-405b-instruct

llm = ChatNVIDIA(model="meta/llama-3.1-405b-instruct", nvidia_api_key=nvapi_key, max_tokens=512)

result = llm.invoke("explain w to change the spair tayar of Honda HRV ho2021?")

print(result.content)

Step 3 - 初始化ai-embed-qa-4向量模型

from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings

embedder = NVIDIAEmbeddings(model="ai-embed-qa-4")

Step 4 - 获取文本数据集

import os

from tqdm import tqdm

from pathlib import Path

# Here we read in the text data and prepare them into vectorstore

ps = os.listdir("./zh_data/")

data = []

sources = []

for p in ps:

    if p.endswith('.pdf'):

        path2file="./zh_data/"+p

        with open(path2file,encoding="utf-8") as f:

            lines=f.readlines()

            for line in lines:

                if len(line)>=1:

                    data.append(line)

                    sources.append(path2file)

Step 5 - 进行一些基本的清理并删除空行

documents=[d for d in data if d != '\n']

len(data), len(documents), data[0]

Step 6a - 将文档处理到 faiss vectorstore 并将其保存到磁盘

# Here we create a vector store from the documents and save it to disk.

from operator import itemgetter

from langchain.vectorstores import FAISS

from langchain_core.output_parsers import StrOutputParser

from langchain_core.prompts import ChatPromptTemplate

from langchain_core.runnables import RunnablePassthrough

from langchain.text_splitter import CharacterTextSplitter

from langchain_nvidia_ai_endpoints import ChatNVIDIA

import faiss

# 只需要执行一次，后面可以重读已经保存的向量存储

# text_splitter = CharacterTextSplitter(chunk_size=400, separator=" ")

# docs = []

# metadatas = []

# for i, d in enumerate(documents):

#     splits = text_splitter.split_text(d)

#     #print(len(splits))

#     docs.extend(splits)

#     metadatas.extend([{"source": sources[i]}] * len(splits))

# store = FAISS.from_texts(docs, embedder , metadatas=metadatas)

# store.save_local('./zh_data/nv_embedding')

Step 6b - 重读之前处理并保存的 Faiss Vectore 存储

# Load the vectorestore back.

Store = FAISS.load_local("./zh_data/nv_embedding", embedder,allow_dangerous_deserialization=True)

Step 7- 提出问题并基于meta/llama-3.1-405b-instruct模型进行RAG检索

retriever = store.as_retriever()

prompt = ChatPromptTemplate.from_messages(

    [

        (

            "system",

            "Answer solely based on the following context:\n<Documents>\n{context}\n</Documents>",

        ),

        ("user", "{question}"),

    ]

)

chain = (

    {"context": retriever, "question": RunnablePassthrough()}

    | prompt

    | llm

    | StrOutputParser()

)

chain.invoke("explain w to change the spair tayar of Honda HRV ho2021?")

### 项目成果与展示

#### 应用场景展示

本项目的智能对话机器人主要应用于车辆导航和系统查询场景，如帮助用户设置导航目的地、了解车辆功能、以及处理常见问题。这些功能极大地提高了用户的驾驶体验和安全性。

#### 功能演示

retriever = store.as_retriever()

prompt = ChatPromptTemplate.from_messages(

    [

        (

            "system",

            "Answer solely based on the following context:\n<Documents>\n{context}\n</Documents>",

        ),

        ("user", "{question}"),

    ]

)

chain = (

    {"context": retriever, "question": RunnablePassthrough()}

    | prompt

    | llm

    | StrOutputParser()

)

chain.invoke("explain how to change the spair tayar of Honda HRV 2021?")

回答：

Changing a spare tire on a Honda HR-V 2021 is a relatively straightforward process. Here's a step-by-step guide to help you through it:

**Important:**

* Make sure you're in a safe location, away from traffic, and turn off the engine.

* Engage the parking brake and put the transmission in Park (automatic) or first gear (manual).

* Turn on the hazard lights.

* Familiarize yourself with the location of the spare tire, tools, and jacking points on your Honda HR-V.

**Location of the spare tire and tools:**

* The spare tire is located under the floor panel in the cargo area.

* The tools ( lug wrench, jack, and wheel chocks) are stored in a bag attached to the cargo area's side panel.

**Step-by-Step Instructions:**

1. **Loosen the lug nuts**:

                    * Before you jack up the vehicle, loosen the lug nuts on the flat tire with the lug wrench.

                    * Turn them counterclockwise until they're loose, but do not remove them yet.

2. **Jack up the vehicle**:

                    * Position the jack under the jacking point nearest to the flat tire.

                    * Raise the jack until the flat tire is off the ground, but not so high that the vehicle is unstable.

3. **Remove the lug nuts and flat tire**:

                    * Completely remove the lug nuts from the wheel hub and set them aside in a safe place.

                    * Carefully pull the flat tire straight off the wheel hub and set it aside.

4. **Install the spare tire**:

                    * Place the spare tire onto the wheel hub and hand tighten the lug nuts.

                    * Make sure the tire is properly seated and centered on the hub.

5. **Tighten the lug nuts**:

                    * Lower the vehicle to the ground using the jack.

                    * Use the lug wrench to tighten the lug nuts in a star pattern (tightening one lug nut a little, then moving to the next one, and so on).

                    * Make sure they're snug, but do not overtighten.

6. **Check the tire pressure**:

                    * Make sure the spare tire is inflated to the recommended pressure, which can be found on the tire's sidewall or in the owner's manual.

7. **Drive carefully**:

                    * Drive carefully to a tire repair shop to have the flat tire repaired or replaced.

                    * Do not drive at high speeds or for an extended period with the spare tire.

**Additional Tips:**

* If you're not comfortable changing the spare tire, call for roadside assistance or seek help from a professional mechanic.

* Make sure to regularly check the spare tire's air pressure and condition to ensure it's ready for use in case of an emergency.

* The spare tire is only intended for temporary use and should be replaced with a full-size tire as soon as possible.

Remember to always refer to your Honda HR-V's owner's manual for specific instructions on how to change a spare tire on your vehicle. If you have any questions or concerns, feel free to ask!

### 项目总结与展望

#### 项目评估

整体上，项目成功实现了预期的功能目标，特别是在车辆导航和功能查询方面表现出色。然而，仍存在复杂使用环境以及复杂的问题下的表现有待提升，

#### 未来方向

未来可以考虑添加语音识别技术，并探索更多的多模态交互场景。同时，可以考虑集成更多的车辆数据源，提供更全面的智能服务。

### 附件与参考资料

车辆手册1）

https://techinfo.honda.com/rjanisis/pubs/OM/AH/AT7A2121NM/enu/AT7A2121NM.PDF

车辆手册2）

https://techinfo.honda.com/rjanisis/pubs/OM/AH/AT7A2121OM/enu/AT7A2121OM.PDF