本地AI助手：在Seeed Studio reComputer Jetson设备上部署Ollama和AnythingLLM

本地AI助手：在Seeed Studio reComputer Jetson设备上部署Ollama和AnythingLLM

简介

本地AI助手是一种在本地硬件和软件上运行的人工智能应用程序，提供智能交互和数据处理功能。它独立于云服务运行，保护用户隐私，同时提供快速响应时间和高度的自定义能力。

在本教程中，我们将演示如何在Jetson设备上使用ollama和AnythingLLM搭建本地知识库。通过利用大语言模型的强大功能，我们旨在提高工作效率。

准备工作

• 配备16GB以上内存的Jetson设备。
• 硬件设备需要预先刷入JetPack 5.1.1操作系统。

注意
在本教程中，我们将使用NVIDIA® Jetson AGX Orin™ 64GB 开发者套件完成以下任务，但你也可以尝试使用其他Jetson设备。

立即购买 🖱️

开始使用

硬件连接

• 将Jetson设备连接到网络、鼠标、键盘和显示器。（当然，你也可以通过本地网络上的SSH远程访问Jetson设备。）

安装和运行Ollama

这里，我们强烈推荐使用jetson-examples在Jetson设备上快速部署Ollama**（教程不易，请给Github点点小星星）**。

步骤1. 在Jetson设备上打开终端，并运行以下命令安装 jetson-examples：

sudo apt install python3-pip
pip3 install jetson-examples

步骤2. 在Jetson设备上使用一条命令部署Ollama：

reComputer run ollama

步骤3. 使用Ollama下载Llama3大模型：

ollama run llama3

请保持此终端处于活跃状态。

安装和运行AnythingLLM

我们可以使用本地Docker设置方便地安装AnythingLLM。

步骤1. 在Jetson上运行AnythingLLM。

重新打开终端并输入以下命令：

docker pull mintplexlabs/anythingllm

export STORAGE_LOCATION=$HOME/anythingllm 
mkdir -p $STORAGE_LOCATION 
touch "$STORAGE_LOCATION/.env" 
docker run -d -p 3001:3001 --cap-add SYS_ADMIN \
    -v ${STORAGE_LOCATION}:/app/server/storage \
    -v ${STORAGE_LOCATION}/.env:/app/server/.env \
    -e STORAGE_DIR="/app/server/storage" \
    mintplexlabs/anythingllm

步骤2. 配置你自己的本地知识库。

在浏览器中访问 http://<jetson-ip>:3001 以访问完整的应用程序。

你可以在Jetson终端上输入 ifconfig 命令查找其IP地址。

按照屏幕上的提示完成AnythingLLM的配置。

请注意，在LLM Perference的配置页面上，我们应选择Ollama的Llama 3.1大语言模型。

最后，导入所需的文档。然后，大语言模型将基于导入的文档回答问题。

import --> click to upload or drag and drop --> select document --> move to Workspace --> save and embed

至此，你已成功搭建了自己的本地知识库。

欲了解更深入的配置教程，请参考此链接。

运行它

在对话框中输入问题，大语言模型将基于知识库提供答案。

story1.txt

Once upon a time in a quaint village nestled between rolling hills, there was a young girl named Eliza who loved to explore the woods behind her home. The forest was a magical place, filled with tall trees that whispered secrets, streams that sang soft melodies, and flowers that seemed to glow under the moonlight.

One sunny morning, Eliza set out on one of her adventures, her heart brimming with excitement. As she wandered deeper into the forest, she discovered a hidden path she had never seen before. The path was lined with shimmering stones that sparkled like stars. Curious and intrigued, Eliza followed it.

After a short walk, the path led her to a magnificent clearing where a majestic oak tree stood in the center. At the base of the tree was a small, ornate door. It was covered in intricate carvings of animals and vines. Eliza, with her heart pounding with both excitement and nervousness, gently pushed the door open.

Inside, she found herself in a cozy, enchanted room. There were shelves lined with books and strange artifacts, and a warm fire crackling in a stone hearth. In the middle of the room, a wise old owl perched on a branch of a large, leafy plant.

The owl looked at Eliza with kind, knowing eyes. “Welcome, young traveler,” it hooted softly. “I am Oliver, the guardian of this magical realm. Few people find their way here. You must have a special heart.”

Eliza’s eyes widened in awe. “What is this place?” she asked.

“This is the Realm of Wonders,” Oliver explained. “It is a place where dreams come to life and where those with pure intentions can find their heart’s true desire.”

Eliza gazed around the room, her curiosity piqued. “What can I do here?”

Oliver smiled. “You can make a wish. But remember, wishes made here come with great responsibility. They have the power to change not just your life but the lives of those around you.”

Eliza thought long and hard. She remembered how her village had been struggling with drought and how her friends and family were suffering. With a determined look, she made her wish.

“I wish for rain to fall upon my village and bring life back to the land.”

Oliver nodded approvingly. “A selfless wish. It will be granted.”

The next morning, as Eliza returned to her village, dark clouds gathered in the sky, and a gentle rain began to fall. The villagers looked up in amazement as the parched earth drank in the life-giving water. The fields began to turn green, and the village flourished once more.

Eliza’s heart swelled with joy as she realized the impact of her wish. The Realm of Wonders had given her the chance to make a difference, and she learned that true magic comes from caring for others.

From that day on, Eliza continued to explore the woods, knowing that the true wonders of life were found in kindness and selflessness.

And so, the village thrived, and Eliza’s adventures became the stuff of legends, reminding everyone that magic, indeed, begins with a kind heart.

效果演示

在此，我们使用配置好的本地个人助手查询我们需要的信息。
Youtube

参考资料

• 一键部署项目: Jetson examples
• Ollama官方:
• anythingllm官方
• 演示效果
• 本文内容翻译自Seeed Studio Wiki

内容贡献者

Seeed Studio–Yu Youjiang